Deep reinforcement learning (DRL) networks currently rival human-level performance in a variety of domains, including object recognition [1]}, speech recognition [2]}, video games [3]}, and complex board games such as Go [4]}. Despite the impressive achievements of DRL, networks fail to adapt to trivial changes of the inputs and goals of the learning task, such as changes to board dimensions and structure [5]}. One potential reason for this shortcoming is that DRL algorithms learn through extensive feedback about the value of specific input-output associations, without any appreciation for the organizing features of the game that govern these associations. In contrast, evidence from cognitive science suggests that humans learn to perform complex tasks through a model-based approach that involves constructing an internal model of the organizing principles or rules of the environment [5]}, [7]}. This form of learning is particularly important in dynamic environments where survival depends on the ability to generalize previous training to novel settings [8]}. While model-based learning algorithms stand to improve the robustness of DRL networks in dynamic environments, it remains a largely unanswered question how this might be achieved.
