Physics 101: Learning Physical Object Properties from Unlabeled Videos
Jiajun Wu, Joseph Lim, Hongyi Zhang, Joshua Tenenbaum and William Freeman
Abstract
We study the problem of learning physical properties of objects from unlabeled videos. Humans can learn basic physical laws when they are very young, which suggests that such tasks may be important goals for computational vision systems. We consider various scenarios: objects sliding down an inclined surface and colliding; objects attached to a spring; objects falling onto various surfaces, etc. Many physical properties like mass, density, and coefficient of restitution influence the outcome of these scenarios, and our goal is to recover them automatically. We have collected 17,408 video clips containing 101 objects of various materials and appearances (shapes, colors, and sizes). Together, they form a dataset, named Physics 101, for studying object-centered physical properties. We propose an unsupervised representation learning model, which explicitly encodes basic physical laws into the structure and use them, with automatically discovered observations from videos, as supervision. Experiments demonstrate that our model can learn physical properties of objects from video. We also illustrate how its generative nature enables solving other tasks such as outcome prediction.
Session
Posters 1
Files
Extended Abstract (PDF, 665K)
Paper (PDF, 8M)
DOI
10.5244/C.30.39
https://dx.doi.org/10.5244/C.30.39
Citation
Jiajun Wu, Joseph Lim, Hongyi Zhang, Joshua Tenenbaum and William Freeman. Physics 101: Learning Physical Object Properties from Unlabeled Videos. In Richard C. Wilson, Edwin R. Hancock and William A. P. Smith, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 39.1-39.12. BMVA Press, September 2016.
Bibtex
@inproceedings{BMVC2016_39,
title={Physics 101: Learning Physical Object Properties from Unlabeled Videos},
author={Jiajun Wu, Joseph Lim, Hongyi Zhang, Joshua Tenenbaum and William Freeman},
year={2016},
month={September},
pages={39.1-39.12},
articleno={39},
numpages={12},
booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
publisher={BMVA Press},
editor={Richard C. Wilson, Edwin R. Hancock and William A. P. Smith},
doi={10.5244/C.30.39},
isbn={1-901725-59-6},
url={https://dx.doi.org/10.5244/C.30.39}
}