Real-time Visual-Inertial Odometry for Event Cameras using Keyframe-based Nonlinear Optimization

Henri Rebecq, Timo Horstschaefer and Davide Scaramuzza

Abstract

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. We propose a novel, accurate tightly-coupled visual-inertial odometry pipeline for such cameras that leverages their outstanding properties to estimate the camera ego-motion in challenging conditions, such as high-speed motion or high dynamic range scenes. The method tracks a set of features (extracted on the image plane) through time. To achieve that, we consider events in overlapping spatio-temporal windows and align them using the current camera motion and scene structure, yielding motion-compensated event frames. We then combine these feature tracks in a keyframe-based, visual-inertial odometry algorithm based on nonlinear optimization to estimate the camera’s 6-DOF pose, velocity, and IMU biases. The proposed method is evaluated quantitatively on the public Event Camera Dataset [19] and significantly outperforms the state-of-the-art [28], while being computationally much more efficient: our pipeline can run much faster than real-time on a laptop and even on a smartphone processor.

Session

Orals - Pose Estimation

Files

Paper (PDF)
Supplementary (PDF)

DOI

10.5244/C.31.16
https://dx.doi.org/10.5244/C.31.16

Citation

Bibtex

            @inproceedings{BMVC2017_16,
                title={Real-time Visual-Inertial Odometry for Event Cameras using Keyframe-based Nonlinear Optimization},
                author={Henri Rebecq, Timo Horstschaefer and Davide Scaramuzza},
                year={2017},
                month={September},
                pages={16.1-16.12},
                articleno={16},
                numpages={12},
                booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
                publisher={BMVA Press},
                editor={Tae-Kyun Kim, Stefanos Zafeiriou, Gabriel Brostow and Krystian Mikolajczyk},
                doi={10.5244/C.31.16},
                isbn={1-901725-60-X},
                url={https://dx.doi.org/10.5244/C.31.16}
            }