NRSfM-Flow: Recovering Non-Rigid Scene Flow from Monocular Image Sequences

Vladislav Golyanik, Aman Shankar Mathur and Didier Stricker

Abstract

Scene flow recovery from monocular image sequences is an emerging field in computer vision. While existing Monocular Scene Flow (MSF) methods extend the classical optical flow formulation to estimate depths/disparities and 3D motion, we propose a framework based on Non-Rigid Structure from Motion (NRSfM) technique „ NRSfM-Flow. Therefore, both problems are formulated in the continuous domain and relation between them is established. To cope with real data, we propose two preprocessing steps for image sequences „ redundancy removal and translation resolution „ which increase quality of reconstructions and speedup computations. In contrast to the existing MSF methods which can cope with non-rigid deformations, our solution makes no strong assumptions about a scene such as known camera motion or camera velocity constancy and can handle occlusions. NRSfM-Flow is qualitatively evaluated on challenging real-world data. Experiments provide evidence that the proposed approach achieves high accuracy and outperforms state of the art in terms of the ability to reconstruct MSF with less prior knowledge about a scene.

Session

Posters 1

Files

Extended Abstract (PDF, 709K)
Paper (PDF, 4M)
Supplemental Materials (ZIP, 19M)

DOI

10.5244/C.30.41
https://dx.doi.org/10.5244/C.30.41

Citation

Bibtex

        @inproceedings{BMVC2016_41,
        	title={NRSfM-Flow: Recovering Non-Rigid Scene Flow from Monocular Image Sequences},
        	author={Vladislav Golyanik, Aman Shankar  Mathur and Didier Stricker},
        	year={2016},
        	month={September},
        	pages={41.1-41.13},
        	articleno={41},
        	numpages={13},
        	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
        	publisher={BMVA Press},
        	editor={Richard C. Wilson, Edwin R. Hancock and William A. P. Smith},
        	doi={10.5244/C.30.41},
        	isbn={1-901725-59-6},
        	url={https://dx.doi.org/10.5244/C.30.41}
        }