Physics 101: Learning Physical Object Properties from Unlabeled Videos

Jiajun Wu, Joseph Lim, Hongyi Zhang, Joshua Tenenbaum and William Freeman

Abstract

We study the problem of learning physical properties of objects from unlabeled videos. Humans can learn basic physical laws when they are very young, which suggests that such tasks may be important goals for computational vision systems. We consider various scenarios: objects sliding down an inclined surface and colliding; objects attached to a spring; objects falling onto various surfaces, etc. Many physical properties like mass, density, and coefficient of restitution influence the outcome of these scenarios, and our goal is to recover them automatically. We have collected 17,408 video clips containing 101 objects of various materials and appearances (shapes, colors, and sizes). Together, they form a dataset, named Physics 101, for studying object-centered physical properties. We propose an unsupervised representation learning model, which explicitly encodes basic physical laws into the structure and use them, with automatically discovered observations from videos, as supervision. Experiments demonstrate that our model can learn physical properties of objects from video. We also illustrate how its generative nature enables solving other tasks such as outcome prediction.

Session

Posters 1

Files

PDF iconExtended Abstract (PDF, 665K)
PDF iconPaper (PDF, 8M)

DOI

10.5244/C.30.39
https://dx.doi.org/10.5244/C.30.39

Citation

Jiajun Wu, Joseph Lim, Hongyi Zhang, Joshua Tenenbaum and William Freeman. Physics 101: Learning Physical Object Properties from Unlabeled Videos. In Richard C. Wilson, Edwin R. Hancock and William A. P. Smith, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 39.1-39.12. BMVA Press, September 2016.

Bibtex

        @inproceedings{BMVC2016_39,
        	title={Physics 101: Learning Physical Object Properties from Unlabeled Videos},
        	author={Jiajun Wu, Joseph Lim, Hongyi Zhang, Joshua Tenenbaum and William Freeman},
        	year={2016},
        	month={September},
        	pages={39.1-39.12},
        	articleno={39},
        	numpages={12},
        	booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
        	publisher={BMVA Press},
        	editor={Richard C. Wilson, Edwin R. Hancock and William A. P. Smith},
        	doi={10.5244/C.30.39},
        	isbn={1-901725-59-6},
        	url={https://dx.doi.org/10.5244/C.30.39}
        }