Simultaneous Inpainting and Super-resolution Using Self-learning

Milind G. Padalkar, Manjunath V. Joshi and Nilay Khatri

Abstract

Past two decades have seen significant advancement in the techniques for scene completion and image super-resolution. Although many of the approaches solve these two problems by searching and processing of similar patches for estimating the unknown pixel values, the two problems have been addressed independently. In applications like creating immersive walkthrough systems or digital reconstruction of invaluable artwork, both inpainting and super-resolution of the given images are the preliminary steps in order to provide better visual experience. The usual practice is to solve these problems independently in a pipelined manner. In this paper we propose a unified framework to perform simultaneous inpainting and super-resolution. We construct dictionaries of image-representative low and high resolution patch pairs from the known regions in the test image and its coarser resolution. Inpainting of the missing pixels is performed using exemplars found by comparing patch details at a finer resolution, where self-learning is used to obtain the finer resolution patches by making use of the constructed dictionaries. The obtained finer resolution patches represent the super-resolved patches in the missing regions. Advantage of our approach when compared to other exemplar based inpainting techniques are (1) additional constraint in the form of finer resolution matching results in better inpainting and (2) inpainting is obtained not only in the given spatial resolution but also at higher resolution leading to super-resolution inpainting. Experiments on natural images show efficacy of the proposed method in comparison to state-of-the-art methods.

Session

Poster 2

Files

Extended Abstract (PDF, 850K)

Paper (PDF, 4M)

Supplemental Materials (ZIP, 9M)

DOI

10.5244/C.29.105
https://dx.doi.org/10.5244/C.29.105

Citation

Milind G. Padalkar, Manjunath V. Joshi and Nilay Khatri. Simultaneous Inpainting and Super-resolution Using Self-learning. In Xianghua Xie, Mark W. Jones, and Gary K. L. Tam, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 105.1-105.12. BMVA Press, September 2015.

Bibtex

@inproceedings{BMVC2015_105,
	title={Simultaneous Inpainting and Super-resolution Using Self-learning},
	author={Milind G. Padalkar and Manjunath V. Joshi and Nilay Khatri},
	year={2015},
	month={September},
	pages={105.1-105.12},
	articleno={105},
	numpages={12},
	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
	publisher={BMVA Press},
	editor={Xianghua Xie, Mark W. Jones, and Gary K. L. Tam},
	doi={10.5244/C.29.105},
	isbn={1-901725-53-7},
	url={https://dx.doi.org/10.5244/C.29.105}
}