Applications of Reinforcement Learning in Finance -- Trading with a Double Deep Q-Network
This paper presents a Double Deep Q-Network algorithm for trading single
assets, namely the E-mini S&P 500 continuous futures contract. We use a proven
setup as the foundation for our environment with multiple extensions. The
features of our trading agent are constantly being expanded to include
additional assets such as commodities, resulting in four models. We also
respond to environmental conditions, including costs and crises. Our trading
agent is first trained for a specific time period and tested on new data and
compared with the long-and-hold strategy as a benchmark (market). We analyze
the differences between the various models and the in-sample/out-of-sample
performance with respect to the environment. The experimental results show that
the trading agent follows an appropriate behavior. It can adjust its policy to
different circumstances, such as more extensive use of the neutral position
when trading costs are present. Furthermore, the net asset value exceeded that
of the benchmark, and the agent outperformed the market in the test set. We
provide initial insights into the behavior of an agent in a financial domain
using a DDQN algorithm. The results of this study can be used for further
development.