Nurses on the Edge: An On-device Human Activity Recognition Framework that Optimizes the Sensor Placement

On-body Sensor-based Human Activity Recognition provides an excellent opportunity to monitor a person’s movement unobtrusively. This is commonly used to provide recommendations about physical activity and similar, while our main motivation was to streamline the time-consuming process of documenting nursing activities. In this paper we lay out the methodology for building activity recognition models, and present a mobile application that provides comprehensive support for it: from data acquisition and annotation, to on-device training and execution of activity recognition in real time. Furthermore, we developed a novel approach to determine the optimal sensor placement to achieve higher classification accuracy through real-time video data conversion into pose estimations. We demonstrate our systems’ various features in a feasibility study. For this purpose, we conducted a study with inertial sensors on nursing activities with ten subjects on 13 activities. We demonstrate that our vision-based approach for the optimal sensor placement achieved a high correspondence with a trained model. Furthermore, we achieved a higher accuracy for a multimodal and on-device trained model approach compared to a single-modality and offline variant. This evaluation showed that the system served essential support in a realistic environment.