loading page

OMSLD: Occlusion-Aware Multi-human Segmentation in Low-resolution Infrared Dynamic Scene
  • Mengni Yang ,
  • Bo Yang ,
  • Haoxiang Shi
Mengni Yang
Author Profile
Haoxiang Shi
Author Profile


Due to the advantages of preserving privacy, low-resolution thermopile infrared array sensor (TPAS), as an alternative to visible light sensor, has gained increasing attention for real-time perception and monitoring of indoor human targets. However, most recent research focus on single individual just because of its low resolution. Additionally, there are many dynamically changing heat sources besides human targets in indoor environments. The switching states, positional movements of interfering heat sources, as well as occlusions between interfering heat sources and humans and among humans themselves, pose a great challenge and bear significance to the task of removing dynamic background and distinguishing occluded human instances in low resolution. This paper proposes a new infrared occluded multi-human segmentation network based on Mask R-CNN and designs a new multi-scale fusion feature pyramid detection network. A hardware platform is constructed based on a TPAS with 24×32 pixels. The improved Mask R-CNN can effectively segment occluded multi-human instances even in low-resolution dynamically changing scenarios. However, there is still possible to segment dynamic indoor heat sources as human targets. To address this issue, this paper introduces an adaptive Gaussian background removal algorithm applying a priori map for multi-human (AGBR-PMM). A combination of these two methods in low-resolution scenes achieves good segmentation of occluded human instances while eliminating interferences from dynamic indoor scenes. This lays the foundation for subsequent human behavior recognition using low-resolution thermopile infrared array sensor.