Occlusion-aware 3D Morphable Face Models

Bernhard Egger, Andreas Schneider, Clemens Blumer, Andreas Forster, Sandro Schönborn and Thomas Vetter

Abstract

We propose a probabilistic occlusion-aware 3D Morphable Face Model adaptation framework for face image analysis based on the Analysis-by-Synthesis setup. In natural images, parts of the face are often occluded by a variety of objects. Such occlusions are a challenge for face model adaptation. We propose to segment the image into face and non-face regions and model them separately. The segmentation and the face model parameters are not known in advance and have to be adapted to the target image. A good segmentation is necessary to obtain a good face model fit and vice-versa. Therefore, face model adaptation and segmentation are solved together using an EM-like procedure. We use a stochastic sampling strategy based on the Metropolis-Hastings algorithm for face model parameter adaptation and a modified Chan-Vese segmentation for face region segmentation. Previous robust methods are limited to homogeneous, controlled illumination settings and tend to fail for important regions such as the eyes or mouth. We propose a RANSAC-based robust illumination estimation technique to handle complex illumination conditions. We do not use any manual annotation and the algorithm is not optimised to any specific kind of occlusion or database. We evaluate our method on a controlled and an ``in the wild'' database.

Session

3D Computer Vision

Files

Extended Abstract (PDF, 2M)
Paper (PDF, 8M)

DOI

10.5244/C.30.64
https://dx.doi.org/10.5244/C.30.64

Citation

Bibtex

        @inproceedings{BMVC2016_64,
        	title={Occlusion-aware 3D Morphable Face Models},
            author={Bernhard Egger, Andreas Schneider, Clemens Blumer, Andreas Forster, Sandro Sch{\"o}nborn and Thomas Vetter},
        	year={2016},
        	month={September},
        	pages={64.1-64.11},
        	articleno={64},
        	numpages={11},
        	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
        	publisher={BMVA Press},
        	editor={Richard C. Wilson, Edwin R. Hancock and William A. P. Smith},
        	doi={10.5244/C.30.64},
        	isbn={1-901725-59-6},
        	url={https://dx.doi.org/10.5244/C.30.64}
        }