Unsupervised Spatio-Temporal Segmentation with Sparse Spectral-Clustering

Mahsa Ghafarianzadeh, Matthew B. Blaschko and Gabe Sibley

In Proceedings British Machine Vision Conference 2014
http://dx.doi.org/10.5244/C.28.9

Abstract

Spatio-temporal cues are powerful sources of information for segmentation in videos. In this work we present an efficient and simple technique for spatio-temporal segmentation that is based on a low-rank spectral clustering algorithm. The complexity of graph-based spatio-temporal segmentation is dominated by the size of the graph, which is proportional to the number of pixels in a video sequence. In contrast to other works, we avoid oversegmenting the images into super-pixels and instead generalize a simple graph based image segmentation. Our graph construction encodes appearance and motion information with temporal links based on optical flow. For large scale data sets naive graph construction is computationally and memory intensive, and has only been achieved previously using a high power compute cluster. We make feasible for the first time large scale graph-based spatio-temporal segmentation on a single core by exploiting the sparsity structure of the problem and a low rank factorization that has strong approximation guarantees. We empirically demonstrate that constructing the low rank approximation using a subset of pixels (30\%-50\%) achieves performance exceeding the state-of-the-art on the Hopkins 155 dataset, while enabling the graph to fit in core memory.

Session

Video and Structure From Motion

Files

Extended Abstract (PDF, 1 page, 350K)

Paper (PDF, 10 pages, 2.8M)

Bibtex File

Presentation

Citation

Mahsa Ghafarianzadeh, Matthew B. Blaschko and Gabe Sibley. Unsupervised Spatio-Temporal Segmentation with Sparse Spectral-Clustering. Proceedings of the British Machine Vision Conference. BMVA Press, September 2014.

BibTex

@inproceedings{BMVC.28.9
	title = {Unsupervised Spatio-Temporal Segmentation with Sparse Spectral-Clustering},
	author = {Ghafarianzadeh, Mahsa and Blaschko, Matthew B. and Sibley, Gabe},
	year = {2014},
	booktitle = {Proceedings of the British Machine Vision Conference},
	publisher = {BMVA Press},
	editors = {Valstar, Michel and French, Andrew and Pridmore, Tony}
	doi = { http://dx.doi.org/10.5244/C.28.9 }
}