loading page

Real-time Hand-Tracking with Skin-Conformal Force Myography based on Wrist-Worn Laser Induced Graphene Strain Sensor Arrays
  • Vinay Kammarchedu,
  • Monther Alsiyabi,
  • Aida Ebrahimi
Vinay Kammarchedu
Materials Research Institute, Center for Atomically Thin Multifunctional Coatings, Department of Electrical Engineering

Corresponding Author:[email protected]

Author Profile
Monther Alsiyabi
Department of Electrical Engineering
Aida Ebrahimi
Department of Biomedical Engineering, The Pennsylvania State University, Materials Research Institute, Center for Atomically Thin Multifunctional Coatings, Department of Electrical Engineering

Abstract

Force myography (FMG) measures the movement of limbs or appendages by monitoring force at their surface to characterize the state of the underlying musculotendinous complex. Compared to electromyography methods, FMG offers a better alternative for monitoring the muscular activity due to direct measurement of force, less sensitivity to skin preparation and skin impedance, and high compatibility with wearable prosthetics and orthotics. Despite significant progress in developing FMG systems, existing devices are still bulky and restrictive to the user or to the placement of the exoskeleton systems. In this work, we develop a wristband integrating an array of ten skin-conformal and wearable strain sensors based on laser induced graphene optimized for continuous measurement of FMG signals. We characterize the device to identify several hand gestures and tasks while simultaneously using an optical camera-based hand tracking system to estimate the joint locations for ground truth generation. We develop machine learning models to predict the gestures as well as specific hand joint angles with a high accuracy (> 90% and > 95%, respectively). We find that sensors that are placed closer to actuation specific anatomical features contribute more towards the high accuracy. We also integrate the sensor array with a wearable readout system that wirelessly transmits the data in real-time, which is used to control a robotic arm as a proof of concept for human-robot interaction applications. The developed skin-conformal FMG device is expected to find wide applications in rehabilitation, sports sciences, and humancomputer interaction, paving the way for low profile prosthetic and orthotic control systems.
28 May 2024Submitted to TechRxiv
04 Jun 2024Published in TechRxiv