loading page

Self-Learned Autonomous Driving at Unsignalized Intersections: A Hierarchical Reinforced Learning Approach for Feasible Decision-Making
  • +3
  • Mohammad Al-Sharman ,
  • Rowan Dempster ,
  • Mohamed A. Daoud ,
  • Mahmoud Nasr ,
  • Derek Rayside ,
  • William Melek
Mohammad Al-Sharman
University of Waterloo

Corresponding Author:[email protected]

Author Profile
Rowan Dempster
Author Profile
Mohamed A. Daoud
Author Profile
Mahmoud Nasr
Author Profile
Derek Rayside
Author Profile
William Melek
Author Profile


Reinforcement learning-based techniques, empowered by deep-structured neural nets, have demonstrated superiority over rule-based methods in terms of making high-level behavioral decisions due to qualities related to handling large state spaces. Nonetheless, their training time, sample efficiency and the feasibility of the learnt behaviors remain key concerns. In this paper, we propose a novel hierarchical reinforcement learning-based decision-making architecture for learning left-turn policies at  unsignalized intersections with feasibility guarantees. The proposed technique is comprised of two layers; a high-level learning-based behavioral planning layer which adopts soft actor-critic principles to learn high-level, non-conservative yet safe, driving behaviors, and a low-level Model Predictive Control (MPC) framework to ensure feasibility of the two-dimensional left-turn maneuver. The high-level layer generates reference signals of velocity and yaw angles for the ego vehicle taking into account safety and collision avoidance with the intersection vehicles, whereas the low-level planning layer solves an optimization problem to track these reference commands taking into account several vehicle dynamic constraints and ride comfort. We validate the proposed decision-making scheme in simulated environments and compare with other model free Reinforcement Learning (RL) baselines. The results demonstrate that the proposed integrated framework possesses better training and navigation capabilities.
2023Published in IEEE Transactions on Intelligent Transportation Systems on pages 1-12. 10.1109/TITS.2023.3285440