Learning Grimaces by Watching TV

Samuel Albanie and Andrea Vedaldi

Abstract

Differently from computer vision systems which require explicit supervision, humans can learn facial expressions by simply observing other humans in their environment. In this paper, we look at the problem of developing similar capabilities in machine vision. As a starting point, we consider the problem of relating facial expressions to objectively measurable events occurring in videos. In particular, we consider a gameshow in which contestants play to win significant sums of money. We extract events affecting the game and corresponding facial expressions objectively and automatically from the videos, obtaining large quantities of labelled data for our study. We also develop, using benchmarks such as FER and SFEW 2.0, state-of-the-art deep neural networks for facial expression recognition, showing that pre-training on face verification data can be highly beneficial for this task. Then, we extend these models to use facial expressions to predict events in videos and learn nameable expressions from them. The dataset and emotion recognition models are available at http://www.robots.ox.ac.uk/~vgg/data/facevalue.

Session

Posters 2

Files

Extended Abstract (PDF, 1M)
Paper (PDF, 3M)

DOI

10.5244/C.30.122
https://dx.doi.org/10.5244/C.30.122

Citation

Bibtex

        @inproceedings{BMVC2016_122,
        	title={Learning Grimaces by Watching TV},
        	author={Samuel Albanie and Andrea Vedaldi},
        	year={2016},
        	month={September},
        	pages={122.1-122.12},
        	articleno={122},
        	numpages={12},
        	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
        	publisher={BMVA Press},
        	editor={Richard C. Wilson, Edwin R. Hancock and William A. P. Smith},
        	doi={10.5244/C.30.122},
        	isbn={1-901725-59-6},
        	url={https://dx.doi.org/10.5244/C.30.122}
        }