The combination of Deep Neural Networks (DNNs) [1]}, [2]} and Reinforcement Learning (RL), often referred to as Deep RL, has allowed artificial agents to solve human-like problems such as playing video games [3]}, [4]}. Despite this impressive achievement, the reliance of Deep RL on DNNs means that it is often criticised for being a 'black-box' approach that is unable to explain the behaviour that it has learnt. The purpose of the present research is to address this criticism by providing a method for generating concise task-level explanations using Deep RL.
