Online Adaptation of Convolutional Neural Networks for Video Object Segmentation
Paul Voigtlaender and Bastian Leibe
Abstract
We tackle the task of semi-supervised video object segmentation, i.e. segmenting
the pixels belonging to an object in a video using the ground truth pixel mask for the
first frame. We build on the recently introduced one-shot video object segmentation
(OSVOS) approach which uses a pretrained network and fine-tunes it on the first frame.
While achieving impressive performance, at test time OSVOS uses the fine-tuned network in unchanged form and is not able to adapt to large changes in object appearance.
To overcome this limitation, we propose Online Adaptive Video Object Segmentation
(OnAVOS) which updates the network online using training examples selected based on
the confidence of the network and the spatial configuration. Additionally, we add a pretraining step based on objectness, which is learned on PASCAL. Our experiments show
that both extensions are highly effective and improve the state of the art on DAVIS to an
intersection-over-union score of 85.
Session
Posters
Files
Paper (PDF)
Supplementary (PDF)
DOI
10.5244/C.31.116
https://dx.doi.org/10.5244/C.31.116
Citation
Paul Voigtlaender and Bastian Leibe. Online Adaptation of Convolutional Neural Networks for Video Object Segmentation. In T.K. Kim, S. Zafeiriou, G. Brostow and K. Mikolajczyk, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 116.1-116.13. BMVA Press, September 2017.
Bibtex
@inproceedings{BMVC2017_116,
title={Online Adaptation of Convolutional Neural Networks for Video Object Segmentation},
author={Paul Voigtlaender and Bastian Leibe},
year={2017},
month={September},
pages={116.1-116.13},
articleno={116},
numpages={13},
booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
publisher={BMVA Press},
editor={Tae-Kyun Kim, Stefanos Zafeiriou, Gabriel Brostow and Krystian Mikolajczyk},
doi={10.5244/C.31.116},
isbn={1-901725-60-X},
url={https://dx.doi.org/10.5244/C.31.116}
}