Importantly, rather than arising from an entirely separate and independent process, model-based learning can be viewed as a companion system that is both informed by and exerts control over the feedback-dependent associations formed through model-free learning [1]}. Behavior becomes “model-based” when lower-level feedback dependent representations are leveraged to construct an internal model of the environmental dynamics responsible for previous observations [2]}, sometimes referred to as a “generative model”. A key difference in the behavioral outcomes of these two forms of learning is flexibility [3]}, [1]}: model-free learning results in habitual actions based on a static cache of associated values whereas model-based learning results in goal-directed actions based on inferred dynamics of the environment. Evidence from human neuroimaging experiments suggests that the shift from model-free to model-based policies is driven by a concomitant shift from BG to prefrontal behavioral control [3]}, [6]}, signaling a shift away from feedback-dependent knowledge to active predictions drawn from the agent’s internal model of the environment.
