In addition to the grid world task, we also assess our approach on the continuous mountain car task. The continuous mountain car task is fundamentally different to the grid world task in that both the state and action space of the agent are continuous. The agent is represented by a car that starts in-between two hills and the goal of the agent is to reach the top of the right-hand hill. However the car's engine has insufficient power to drive directly up the hill and must therefore gather momentum in order to reach the top. The state of the agent is represented by two continuous values; one value represents the position of the car (\([-1.2, 0.6]\) ) and the other represents the velocity of the car (\([-0.07, 0.07]\) ). The action space of the agent is one dimensional and represents the force to be applied to the car (\([-1.0, 1.0]\) ) on that time-step. With respect to the reward function, the agent receives a positive reward of +100 for reaching the top of the hill as well as a negative reward that is equal to the squared sum of the actions chosen. The episode terminates when the car reaches the top of the hill or the episode length is greater than 1000. For all simulations we used the implementation of continuous mountain car provided by OpenAI gym [1]}. An example of the task can be seen in Figure REF .
