Cascaded Boundary Regression for Temporal Action Detection

Jiyang Gao, Zhenheng Yang and Ram Nevatia

Abstract

Temporal action detection in long videos is an important problem. State-of-the-art methods address this problem by applying action classifiers on sliding windows. Although sliding windows may contain an identifiable portion of the actions, they may not necessarily cover the entire action instance, which would lead to inferior performance. We adapt a two-stage temporal action detection pipeline with Cascaded Boundary Regression (CBR) model. Class-agnostic proposals and specific actions are detected respec- tively in the first and the second stage. CBR uses temporal coordinate regression to refine the temporal boundaries of the sliding windows. The salient aspect of the refinement process is that, inside each stage, the temporal boundaries are adjusted in a cascaded way by feeding the refined windows back to the system for further boundary refinement. We test CBR on THUMOS-14 and TVSeries, and achieve state-of-the-art performance on both datasets. The performance gain is especially remarkable under high IoU thresholds, e.g. map@tIoU=0.5 on THUMOS-14 is improved from 19.0% to 31.0%

Session

Posters

Files

Paper (PDF)

DOI

10.5244/C.31.52
https://dx.doi.org/10.5244/C.31.52

Citation

Bibtex

            @inproceedings{BMVC2017_52,
                title={Cascaded Boundary Regression for Temporal Action Detection},
                author={Jiyang Gao, Zhenheng Yang and Ram Nevatia},
                year={2017},
                month={September},
                pages={52.1-52.11},
                articleno={52},
                numpages={11},
                booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
                publisher={BMVA Press},
                editor={Tae-Kyun Kim, Stefanos Zafeiriou, Gabriel Brostow and Krystian Mikolajczyk},
                doi={10.5244/C.31.52},
                isbn={1-901725-60-X},
                url={https://dx.doi.org/10.5244/C.31.52}
            }