Video Segmentation by Non-Local Consensus voting

Alon Faktor and Michal Irani

In Proceedings British Machine Vision Conference 2014
http://dx.doi.org/10.5244/C.28.21

Abstract

We address the problem of Foregrond/Background segmentation of ``unconstrained'' video. By ``unconstrained'' we mean that the moving objects and the background scene may be highly non-rigid (e.g., waves in the sea); the camera may undergo a complex motion with 3D parallax; moving objects may suffer from motion blur, large scale and illumination changes, etc. Most existing segmentation methods fail on such unconstrained videos, especially in the presence of highly non-rigid motion and low resolution. We propose a computationally efficient algorithm which is able to produce accurate results on a large variety of unconstrained videos. This is obtained by casting the video segmentation problem as a voting scheme on the graph of co-occurring regions in the video sequence. We start from crude saliency votes at each pixel, and iteratively correct those votes by `consensus voting' of co-occurring regions across the video sequence. The power of our consensus voting comes from the non-locality of the region co-occurrence, both in space and in time -- enabling propagation of diverse and rich information across the entire video sequence. Qualitative and quantitative experiments indicate that our approach outperforms current state-of-the-art methods.

Session

Segmentation and Object Detection

Files

Extended Abstract (PDF, 1 page, 515K)
Paper (PDF, 12 pages, 1.5M)
Bibtex File

Presentation

Citation

BibTex