loading page

On CVaR-Based Reinforcement Learning in Quantitative Investment
  • Ali Alameer ,
  • Khaled Alshehri
Ali Alameer
King Fahd University of Petroleum and Minerals

Corresponding Author:[email protected]

Author Profile
Khaled Alshehri
Author Profile


We propose Conditional Value-at-Risk (CVaR) investment agents to solve the problems of single asset trading and assets allocation under the Direct Reinforcement Learning framework. We propose two convex CVaR-based agents, the CVaR-constrained and the unconstrained CVaR-sensitive. Convexity allows conveniently implementing incremental learning, leading to an adaptive investing agent at an efficient computational cost with the merit of guaranteed policy convergence. Our experiments with frictional investment under various markets reveal the CVaR-constrained potency in improving investment return per unit of risk. The unconstrained CVaR-sensitive agent, on the other hand, exhibits robustness in handling intense market pullbacks, with both CVaR-based agents showing superior risk management to a risk-insensitive one. Our approach also showed superiority over state-of-the-art methods, demonstrating the potential of CVaR-based RL investment models. We finally show how our agents are extendable to learn investing under the most general investment problem of optimizing a multi-asset portfolio.