Following DQN, work on deep RL has progressed and expanded at a remarkable pace. Deep RL has been scaled up to highly complex game domains ranging from Dota [1]} to StarCraft II [2]} to capture-the-flag [3]}. Novel architectures have been developed that support effective deep RL in tasks requiring detailed long-term memory [4]}, [5]}. Deep RL has been integrated with model-based planning, resulting in super-human play in complex games including chess and go [6]}, [7]}, [8]}, [9]}. Further, methods have been developed to allow deep RL to tackle difficult problems in continuous motor control, including simulations of soccer and gymnastics [10]}, [11]}, and robotics problems such as in-hand manipulation of a Rubik's cube [12]}. We review some of these developments in greater detail below, as part of a larger consideration of what implications deep RL may have for neuroscience, the topic to which we now turn.
