Combining Shape from Shading and Stereo: A Variational Approach for the Joint Estimation of Depth, Illumination and Albedo

Daniel Maurer, Yong Chul Ju, Michael Breuß and Andres Bruhn

Abstract

Shape from shading (SfS) and stereo are two fundamentally different strategies for image-based 3-D reconstruction. While approaches for SfS infer the depth solely from pixel intensities, methods for stereo are based on a matching process that establishes correspondences across images. In this paper we propose a joint variational method that combines the advantages of both strategies. By integrating recent stereo and SfS models into a single minimisation framework, we obtain an approach that exploits shading information to improve upon the reconstruction quality of robust stereo methods. To this end, we fuse a Lambertian SfS approach with a robust stereo model and supplement the resulting energy functional with a detail-preserving anisotropic second-order smoothness term. Moreover, we extend the novel model in such a way that it jointly estimates depth, albedo and illumination. This in turn makes it applicable to objects with non-uniform albedo as well as to scenes with unknown illumination. Experiments for synthetic and real-world images show the advantages of our combined approach: While the stereo part overcomes the albedo-depth ambiguity inherent to all SfS methods, the SfS part improves the degree of details of the reconstruction compared to pure stereo methods.

Session

Recognition and Physics-based vision

Files

Extended Abstract (PDF, 8M)
Paper (PDF, 16M)
Supplemental Materials (ZIP, 13M)

DOI

10.5244/C.30.76
https://dx.doi.org/10.5244/C.30.76

Citation

Bibtex

        @inproceedings{BMVC2016_76,
        	title={Combining Shape from Shading and Stereo: A Variational Approach for the Joint Estimation of Depth, Illumination and Albedo},
            author={Daniel Maurer, Yong Chul Ju, Michael Breu{\ss} and Andres Bruhn},
        	year={2016},
        	month={September},
        	pages={76.1-76.14},
        	articleno={76},
        	numpages={14},
        	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
        	publisher={BMVA Press},
        	editor={Richard C. Wilson, Edwin R. Hancock and William A. P. Smith},
        	doi={10.5244/C.30.76},
        	isbn={1-901725-59-6},
        	url={https://dx.doi.org/10.5244/C.30.76}
        }