Exploiting Image-trained CNN Architectures for Unconstrained Video Classification

Shengxin Zha, Florian Luisier, Walter Andrews, Nitish Srivastava and Ruslan Salakhutdinov

Abstract

We conduct an in-depth exploration of different strategies for doing event detection in videos using convolutional neural networks (CNNs) trained for image classification. We study different ways of performing spatial and temporal pooling, feature normalization, choice of CNN layers as well as choice of classifiers. Making judicious choices along these dimensions led to a very significant increase in performance over more naive approaches that have been used till now. We evaluate our approach on the challenging TRECVID MED'14 dataset with two popular CNN architectures pretrained on ImageNet. On this MED'14 dataset, our methods, based entirely on image-trained CNN features, can outperform several state-of-the-art non-CNN models. Our proposed late fusion of CNN- and motion-based features can further increase the mean average precision (mAP) on MED'14 from 34.95% to 38.74%. The fusion approach yields 89.6% classification accuracy on the challenging UCF-101 dataset.

Session

Poster 1

Files

Extended Abstract (PDF, 478K)
Paper (PDF, 964K)
Supplemental Materials (ZIP, 469K)

DOI

10.5244/C.29.60
https://dx.doi.org/10.5244/C.29.60

Citation

Bibtex

@inproceedings{BMVC2015_60,
	title={Exploiting Image-trained CNN Architectures for Unconstrained Video Classification},
	author={Shengxin Zha and Florian Luisier and Walter Andrews and Nitish Srivastava and Ruslan Salakhutdinov},
	year={2015},
	month={September},
	pages={60.1-60.13},
	articleno={60},
	numpages={13},
	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
	publisher={BMVA Press},
	editor={Xianghua Xie, Mark W. Jones, and Gary K. L. Tam},
	doi={10.5244/C.29.60},
	isbn={1-901725-53-7},
	url={https://dx.doi.org/10.5244/C.29.60}
}