loading page

Video-based activity recognition for automated motor assessment of Parkinson's disease
  • +3
  • Grzegorz Sarapata ,
  • Gareth Morinan ,
  • Yuriy Dushin ,
  • Bernhard Kainz ,
  • Joshua Ong ,
  • Jonathan O'Keeffe
Grzegorz Sarapata
Machine Medicine Technologies

Corresponding Author:[email protected]

Author Profile
Gareth Morinan
Author Profile
Yuriy Dushin
Author Profile
Bernhard Kainz
Author Profile
Joshua Ong
Author Profile
Jonathan O'Keeffe
Author Profile


Over the last decade, video-enabled mobile devices have become almost ubiquitous, while advances in markerless pose estimation allow an individual’s body position to be tracked across the frames of a video. Previous work by this and other groups has shown that pose-extracted kinematic features can be used to reliably measure motor impairment in Parkinson’s disease (PD). This presents the prospect of developing an asynchronous and scalable, video-based assessment of motor dysfunction. Crucial to this endeavour is the ability to automatically recognise the class of an action being performed, without which manual labelling is required.
Representing the evolution of body joint locations as a spatio-temporal graph, we implement a deep-learning model for frame-level classification of activities performed according to part 3 of the Movement Disorder Society Unified PD Rating Scale (MDS-UPDRS). We train and validate this system using a dataset of n=7220 video clips, recorded at 5 independent sites.
This approach achieves human-level performance in classifying and labelling periods of activity within monocular video clips. Our framework could support clinical workflows and patient care at scale through applications such as quality monitoring of clinical data collection, automated labelling of video streams, or a module within a remote self-assessment system.
Oct 2023Published in IEEE Journal of Biomedical and Health Informatics volume 27 issue 10 on pages 5032-5041. 10.1109/JBHI.2023.3298530