loading page

Deep Learning in Surgical Workflow Analysis: A Review of Phase and Step Recognition
  • +3
  • Kubilay Can Demir ,
  • Hannah Schieber ,
  • Daniel Roth ,
  • Andreas Maier ,
  • Seung Hee Yang ,
  • Tobias Weise
Kubilay Can Demir
Friedrich-Alexander Universitat Erlangen-Nurnberg, Friedrich-Alexander-Universitat Erlangen-Nurnberg, Friedrich-Alexander-Universitat Erlangen-Nurnberg

Corresponding Author:[email protected]

Author Profile
Hannah Schieber
Author Profile
Daniel Roth
Author Profile
Andreas Maier
Author Profile
Seung Hee Yang
Author Profile
Tobias Weise
Author Profile

Abstract

Objective: In the last two decades, there has been a growing interest in exploring surgical procedures with statistical models to analyze operations at different semantic levels. This information is necessary for developing context-aware intelligent systems, which can assist the physicians during operations, evaluate procedures afterward or help the management team to effectively utilize the operating room. The objective is to extract reliable patterns from surgical data for the robust estimation of surgical activities performed during operations. The purpose of this paper is to review the state-of-the-art deep learning methods that have been published after 2018 for analyzing surgical workflows, with a focus on phase and step recognition. Methods: Three databases, IEEE Xplore, Scopus, and PubMed were searched, and additional studies are added through a manual search. After the database search, 343 studies were screened and a total of 44 studies are selected for this review. Conclusion: The use of temporal information is essential for identifying the next surgical action. Contemporary methods used mainly RNNs, hierarchical CNNs, and Transformers to preserve long-distance temporal relations. The lack of large publicly available datasets for various procedures is a great challenge for the development of new and robust models. As supervised learning strategies are used to show proof-of-concept, self-supervised, semi-supervised, or active learning methods are used to mitigate dependency on annotated data. Significance: The present study provides a comprehensive review of recent methods in surgical workflow analysis, summarizes commonly used architectures, datasets, and discusses challenges.
2023Published in IEEE Journal of Biomedical and Health Informatics on pages 1-14. 10.1109/JBHI.2023.3311628