TechRxiv
CVaR_Based_RL__TNNLS_.pdf (1.08 MB)

On CVaR-Based Reinforcement Learning in Quantitative Investment

Download (1.08 MB)
preprint
posted on 2022-06-06, 14:19 authored by Ali AlameerAli Alameer, Khaled AlshehriKhaled Alshehri

We propose Conditional Value-at-Risk (CVaR) investment agents to solve the problems of single asset trading and assets allocation under the Direct Reinforcement Learning framework. We propose two convex CVaR-based agents, the CVaR-constrained and the unconstrained CVaR-sensitive. Convexity allows conveniently implementing incremental learning, leading to an adaptive investing agent at an efficient computational cost with the merit of guaranteed policy convergence. Our experiments with frictional investment under various markets reveal the CVaR-constrained potency in improving investment return per unit of risk. The unconstrained CVaR-sensitive agent, on the other hand, exhibits robustness in handling intense market pullbacks, with both CVaR-based agents showing superior risk management to a risk-insensitive one. Our approach also showed superiority over state-of-the-art methods, demonstrating the potential of CVaR-based RL investment models. We finally show how our agents are extendable to learn investing under the most general investment problem of optimizing a multi-asset portfolio.


History

Email Address of Submitting Author

ali92eee@gmail.com

ORCID of Submitting Author

https://orcid.org/0000-0001-9651-6033

Submitting Author's Institution

King Fahd University of Petroleum and Minerals

Submitting Author's Country

  • Saudi Arabia