In addition to driving consolidation, memory maintenance and retrieval in the brain is also used for online decision-making [1]}, [2]}, [3]}, [4]}. In deep RL, two kinds of memory serve this function. First, `episodic' memory systems read and write long-term storage slots [5]}, [6]}, [7]}. One interesting aspect of these systems is that they allow relatively easy analysis of what information is being stored and retrieved at each timestep [8]}, [9]}, inviting comparisons to neural data. Second, recurrent neural networks store information in activations, in a manner similar to what is referred to in neuroscience as working memory maintenance. The widely used `LSTM' and `GRU' architectures use learnable gating to forget or retain task-relevant information, reminiscent of similar mechanisms which have been proposed to exist in the brain [10]}, [11]}.
