One particularly intriguing finding from deep RL is that there are circumstances under which processes resembling model-based RL may emerge spontaneously within systems trained using model-free RL algorithms [1]}, [2]}. The neuroscientific implications of this `model-free planning' have already been studied in a preliminary way [3]}, but it deserves further investigation. Intriguingly, model-based behavior is also seen in RL systems that employ a particular form of predictive code, referred to as the ‘successor representation’ [4]}, [5]}, [6]}, [7]}, suggesting one possible mechanism through which model-free planning might arise.
