loading page

Closed-Loop Deep Brain Stimulation with Reinforcement Learning and Neural Simulation
  • +1
  • Chia-Hung Cho,
  • Chii-Wann Lin,
  • Pin-Jui Huang,
  • Meng-Chao Chen
Chia-Hung Cho

Corresponding Author:[email protected]

Author Profile
Chii-Wann Lin

Corresponding Author:[email protected]

Author Profile
Pin-Jui Huang
Meng-Chao Chen


Objective: Deep Brain Stimulation (DBS) is effective for movement disorders, particularly Parkinson's disease (PD). However, a closed-loop DBS system using reinforcement learning (RL) for automatic parameter tuning, offering enhanced energy efficiency and the effect of thalamus restoration, is yet to be developed for clinical and commercial applications. Methods: In this research, we instantiate a basal ganglia-thalamic (BGT) model and design it as an interactive environment suitable for RL models. Four finely tuned RL agents based on different frameworks, namely Soft Actor-Critic (SAC), Twin Delayed Deep Deterministic Policy Gradient (TD3), Proximal Policy Optimization (PPO), and Advantage Actor-Critic (A2C), are established for further comparison. Results: Within the implemented RL architectures, the optimized TD3 demonstrates a significant 67% reduction in average power dissipation when compared to the open-loop system while preserving the normal response of the simulated BGT circuitry. As a result, our method mitigates thalamic error responses under pathological conditions and prevents overstimulation. Significance: In summary, this study introduces a novel approach to implementing an adaptive parameter-tuning closed-loop DBS system. Leveraging the advantages of TD3, our proposed approach holds significant promise for advancing the integration of RL applications into DBS systems, ultimately optimizing therapeutic effects in future clinical trials.
27 Jan 2024Submitted to TechRxiv
29 Jan 2024Published in TechRxiv