Towards real-time and energy efficient Siamese tracking - a
hardware-software approach
Abstract
Siamese trackers have been among the state-of-the-art solutions in each
Visual Object Tracking (VOT) challenge over the past few years. However,
with great accuracy comes great computational complexity: to achieve
real-time processing, these trackers have to be massively parallelised
and are usually run on high-end GPUs. Easy to implement, this approach
is energy consuming, and thus cannot be used in many low-power
applications.
To overcome this, one can use energy-efficient embedded devices, such as
heterogeneous platforms joining the ARM processor system with
programmable logic (FPGA). In this work, we propose a hardware-software
implementation of the well-known fully connected Siamese tracker
(SiamFC). We have developed a quantised Siamese network for the FINN
accelerator, using algorithm-accelerator co-design, and performed design
space exploration to achieve the best efficiency-to-energy ratio
(determined by FPS and used resources). For our network, running in the
programmable logic part of the Zynq UltraScale+ MPSoC ZCU104, we
achieved the processing of almost 50 frames-per-second with tracker
accuracy on par with its floating point counterpart, as well as the
original SiamFC network. The complete tracking system, implemented in
ARM with the network accelerated on FPGA, achieves up to 17 fps. These
results bring us towards bridging the gap between the highly accurate
but energy-demanding algorithms and energy-efficient solutions ready to
be used in low-power, edge systems.