Event Fisher Vectors: Robust Encoding Visual Diversity of Visual Streams

Markus Nagel, Thomas Mensink and Cees G. M. Snoek

Abstract

In this paper we focus on event recognition in visual image streams. More specifically, we aim to construct a compact representation which encodes the diversity of the visual stream from just a few observations. For this purpose, we introduce the Event Fisher Vector, a Fisher Kernel based representation to describe a collection of images or the sequential frames of a video. We explore different generative models beyond the Gaussian mixture model as underlying probability distribution. First, the Student's-t mixture model which captures the heavy tails of the small sample size of a collection of images. Second, Hidden Markov Models to explicitly capture the temporal ordering of the observations in a stream. For all our models we derive analytical approximations of the Fisher information matrix, which significantly improves recognition performance. We extensively evaluate the properties of our proposed method on three recent datasets for event recognition in photo collections and web videos, leading to an efficient compact image representation which achieves state-of-the-art performance on all these datasets.

Session

Action and Event

Files

Extended Abstract (PDF, 779K)
Paper (PDF, 835K)

DOI

10.5244/C.29.178
https://dx.doi.org/10.5244/C.29.178

Citation

Bibtex

@inproceedings{BMVC2015_178,
	title={Event Fisher Vectors: Robust Encoding Visual Diversity of Visual Streams},
	author={Markus Nagel and Thomas Mensink and Cees G. M. Snoek},
	year={2015},
	month={September},
	pages={178.1-178.12},
	articleno={178},
	numpages={12},
	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
	publisher={BMVA Press},
	editor={Xianghua Xie, Mark W. Jones, and Gary K. L. Tam},
	doi={10.5244/C.29.178},
	isbn={1-901725-53-7},
	url={https://dx.doi.org/10.5244/C.29.178}
}