On the topic of memory, arguably one of the most important in neuroscience, deep RL once again opens up fascinating new questions and highlights novel computational possibilities. In particular, deep RL provides a computational setting in which to investigate how memory can support reward-based learning and decision making, a topic which has been of growing interest in neuroscience [1]}, [2]}.
The first broadly successful deep RL models relied on experience replay [3]}, wherein past experiences are stored, and intermittently used alongside new experiences to drive learning. This has an intriguing similarity to the replay events observed in hippocampus and elsewhere, and indeed was inspired by this phenomenon and its suspected role in memory consolidation [4]}, [5]}. While early deep RL systems replayed experience uniformly, replay in the brain is not uniform [6]}, [2]}, [8]}, [9]}, and non-uniformity has been explored in machine learning as a way to enhance learning [10]}.
