Assessing thermal imagery integration into object detection methods on
air-based collection platforms
Abstract
Object detection models commonly focus on utilizing the visible spectrum
via Red-Green-Blue (RGB) imagery. Due to various limitations with this
approach in low visibility settings, there is growing interest in fusing
RGB with thermal long wave infrared (LWIR) (7.5 - 13.5 µm) images to
increase object detection performance. However, we still lack baseline
performance metrics evaluating RGB, LWIR and LWIR-RGB fused object
detection machine learning models, especially from air-based platforms.
This study undertakes such an evaluation finding that a blended RGB-LWIR
model generally exhibits superior performance compared to traditional
RGB or LWIR approaches. For example, an RGB-LWIR blend only performed
1-5% behind the RGB approach in predictive power across various
altitudes and periods of clear visibility. Yet, RGB fusion with a
thermal signature overlayed provides edge redundancy and edge emphasis,
both which are vital in supporting edge detection machine learning
algorithms. This approach has the ability to improve object detection
performance for a range of use cases in industrial, consumer,
government, and military applications. Finally, this research
additionally contributes a novel open labeled training dataset of 6,300
images for RGB, LWIR, and RGB-LWIR fused imagery, collected from
air-based platforms, enabling further multispectral machine-driven
object detection research.