Balancing Profit, Risk, and Sustainability for Portfolio Management
Stock portfolio optimization is the process of continuous reallocation of
funds to a selection of stocks. This is a particularly well-suited problem for
reinforcement learning, as daily rewards are compounding and objective
functions may include more than just profit, e.g., risk and sustainability. We
developed a novel utility function with the Sharpe ratio representing risk and
the environmental, social, and governance score (ESG) representing
sustainability. We show that a state-of-the-art policy gradient method -
multi-agent deep deterministic policy gradients (MADDPG) - fails to find the
optimum policy due to flat policy gradients and we therefore replaced gradient
descent with a genetic algorithm for parameter optimization. We show that our
system outperforms MADDPG while improving on deep Q-learning approaches by
allowing for continuous action spaces. Crucially, by incorporating risk and
sustainability criteria in the utility function, we improve on the
state-of-the-art in reinforcement learning for portfolio optimization; risk and
sustainability are essential in any modern trading strategy and we propose a
system that does not merely report these metrics, but that actively optimizes
the portfolio to improve on them.