You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video
In Proceedings British Machine Vision Conference 2014
http://dx.doi.org/10.5244/C.28.30
Abstract
We present a fully unsupervised approach for i) discovery of task-relevant objects and ii) how these objects have been used. Given egocentric video from multiple operators, the approach can discover objects with which the users interact, both static objects such as a coffee machine as well as movable ones such as a cup. Importantly, the common modes of interaction for discovered objects are also found. We investigate using appearance, position, motion and attention, and present results using each and a combination of relevant features. Results show that the method is capable of discovering 95% of task-relevant objects on a variety of daily tasks such as initialising a printer, preparing a coffee or setting up a gym machine. In addition, the approach enables the automatic generation of guidance video on how these objects have been used before.
Session
Image Classification
Files
Extended Abstract (PDF, 1 page, 1.2M)Paper (PDF, 13 pages, 2.8M)
Supplemental Materials (ZIP, 9.8M)
Bibtex File
Presentation
Citation
Dima Damen, Teesid Leelasawassuk, Osian Haines, Andrew Calway, and Walterio Mayol-Cuevas. You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video. Proceedings of the British Machine Vision Conference. BMVA Press, September 2014.
BibTex
@inproceedings{BMVC.28.30
title = {You-Do, I-Learn: Discovering Task Relevant Objects and their Modes of Interaction from Multi-User Egocentric Video},
author = {Damen, Dima and Leelasawassuk, Teesid and Haines, Osian and Calway, Andrew and Mayol-Cuevas, Walterio},
year = {2014},
booktitle = {Proceedings of the British Machine Vision Conference},
publisher = {BMVA Press},
editors = {Valstar, Michel and French, Andrew and Pridmore, Tony}
doi = { http://dx.doi.org/10.5244/C.28.30 }
}