Better Together: Joint Reasoning for Non-rigid 3D Reconstruction with Specularities and Shading

Qi Liu-Yin, Rui Yu, Lourdes Agapito, Andrew Fitzgibbon and Chris Russell

Abstract

In this paper, we demonstrate the use of shape-from-shading (SFS) to improve both the quality and the robustness of 3D reconstruction of dynamic objects captured by a single camera. Unlike previous approaches that made use of SFS~as a post-processing step, we offer a principled integrated approach that solves dynamic object tracking and reconstruction and SFS~as a single unified cost function. Moving beyond Lambertian SFS, we propose a general approach that models both specularities and shading while simultaneously tracking and reconstructing general dynamic objects. Solving these problems jointly prevents the kinds of tracking failures which can not be recovered from by pipeline approaches. We show state-of-the-art results both qualitatively and quantitatively.

Session

Posters 1

Files

Extended Abstract (PDF, 182K)
Paper (PDF, 959K)

DOI

10.5244/C.30.42
https://dx.doi.org/10.5244/C.30.42

Citation

Bibtex

        @inproceedings{BMVC2016_42,
        	title={Better Together: Joint Reasoning for Non-rigid 3D Reconstruction with Specularities and Shading},
        	author={Qi Liu-Yin, Rui Yu, Lourdes Agapito, Andrew Fitzgibbon and Chris Russell},
        	year={2016},
        	month={September},
        	pages={42.1-42.12},
        	articleno={42},
        	numpages={12},
        	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
        	publisher={BMVA Press},
        	editor={Richard C. Wilson, Edwin R. Hancock and William A. P. Smith},
        	doi={10.5244/C.30.42},
        	isbn={1-901725-59-6},
        	url={https://dx.doi.org/10.5244/C.30.42}
        }