In this section we briefly describe the key components of Complementary Temporal Difference Learning (CTDL) (Figure REF ), however for an in-depth description of CTDL please see [1]}. The fundamental components of CTDL are a Deep Neural Network (DNN), which represents a neocortical learning system, and a Self-Organizing Map (SOM), which represents a hippocampal learning system. The DNN can be any standard Deep RL algorithm involving a value function, such as a Deep Q-Network [2]} or Advantage Actor-Critic [3]}. However the final value prediction is not solely based on the prediction of the DNN but is instead a weighted average of the predictions from the DNN and the SOM.
<FIGURE>