High Speed Hybrid Object Tracking Algorithm using Image and Event Data

Event cameras are novel, bio-inspired, imagers that output per-pixel brightness changes at ~1us latency. Besides their high response rates, they offer numerous advantages over conventional cameras, such as no motion blur, high dynamic range, and low power consumption. However, event cameras suffer from some limitations such as lacking the intensity information that regular cameras provide. In this paper, we present a hybrid object tracking algorithm that leverages both images and events, thus, providing a complementary approach that utilizes some of the advantages of both imaging types. Our tracking algorithm detects the objects in the image frames, then tracks objects in the blind time between consecutive frames using per-object event masks extracted from the event data. Moreover, we set up a data collection experiment to evaluate and analyze our algorithm’s performance using Dynamic and Active-Pixel Vision Sensor (DAVIS), which combines a monochrome camera as well as an event-based sensor using the same pixel array. Results show that our tracking algorithm can reach up to 500 Hz tracking rates based on a standard image framerate of 24 Hz and asynchronous event-data data collected by the hybrid camera.