Kinematic-Layout-aware Random Forests for Depth-based Action Recognition

Seungryul Baek, Zhiyuan Shi, Masato Kawade and Tae-Kyun Kim

Abstract

This paper tackles the problem of 24 hours monitoring patient actions in a ward such as “lying on the bed”, “stretching an arm out of the bed” and “falling out of the bed”. In the concerned scenario, 3D geometric information (e.g. relations between scene layouts and body kinematics) is important to reveal the actions; however securing them at testing itself is a challenging problem. Especially in our data, securing human skeletal joints at testing time is not easy due to unique and diverse human posture. To address the problem, we propose a kinematic-layout-aware random forest considering the geometry between scene layouts and skeletons (i.e. kinematic-layout), secured in the offline manner, in the training of forests to maximize the discriminant power of depth appearance. We inte- grate the kinematic-layout in the split criteria of random forests to guide the learning process by 1) measuring the usefulness of kinematic-layout information and switching the use of kinematic-layout, and 2) implicitly closing the gap between two distributions obtained by the kinematic-layout and the appearance, if the kinematic-layout appears useful. Experimental evaluations on our new dataset (PATIENT) demonstrate that our method outperforms various state-of-the-arts for this problem. We have also demon- strated accuracy improvements by applying our method to conventional single-view and cross-view action recognition datasets (e.g. CAD-60, UWA3D Multiview Activity II).

Session

Orals - Pose Estimation

Files

Paper (PDF)
Supplementary (PDF)

DOI

10.5244/C.31.13
https://dx.doi.org/10.5244/C.31.13

Citation

Bibtex

            @inproceedings{BMVC2017_13,
                title={Kinematic-Layout-aware Random Forests for Depth-based Action Recognition},
                author={Seungryul Baek, Zhiyuan Shi, Masato Kawade and Tae-Kyun Kim},
                year={2017},
                month={September},
                pages={13.1-13.15},
                articleno={13},
                numpages={15},
                booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
                publisher={BMVA Press},
                editor={Tae-Kyun Kim, Stefanos Zafeiriou, Gabriel Brostow and Krystian Mikolajczyk},
                doi={10.5244/C.31.13},
                isbn={1-901725-60-X},
                url={https://dx.doi.org/10.5244/C.31.13}
            }