In AlphaGo, the action-outcome model used in planning is hand-coded. Still more interesting, from a neuroscientific point of view [1]}, is recent work in which model-based RL relies on models learned from experience [2]}, [3]}, [4]}. Although these algorithms have achieved great success in some domains, a key open question is whether systems can learn to capture transition dynamics at a high-level of abstraction (“If I throw a rock at that window, it will shatter”) rather than being tied to detailed predictions about perceptual observations (predicting where each shard would fall) [5]}, [6]}.
