On CVaR-Based Reinforcement Learning in Quantitative Investment

We propose Conditional Value-at-Risk (CVaR) investment agents to solve the problems of single asset trading and assets allocation under the Direct Reinforcement Learning framework. We propose two convex CVaR-based agents, the CVaR-constrained and the unconstrained CVaR-sensitive. Convexity allows conveniently implementing incremental learning, leading to an adaptive investing agent at an efficient computational cost with the merit of guaranteed policy convergence. Our experiments with frictional investment under various markets reveal the CVaR-constrained potency in improving investment return per unit of risk. The unconstrained CVaR-sensitive agent, on the other hand, exhibits robustness in handling intense market pullbacks, with both CVaR-based agents showing superior risk management to a risk-insensitive one. Our approach also showed superiority over state-of-the-art methods, demonstrating the potential of CVaR-based RL investment models. We finally show how our agents are extendable to learn investing under the most general investment problem of optimizing a multi-asset portfolio.