One reason for the paucity of model-based approaches to deep learning is that DRL agents are typically developed to solve highly complex tasks, thereby precluding any straightforward process for exploring possible internal models of the environment[1]}. One way to facilitate development of deep model-based learning agents is to identify simpler testing environments that effectively reduce the number of available features from which internal models can be constructed. A recent paper [2]} highlighted several advantages of simplified environments for evaluating the safety and robustness of DRL agents, showing that simple "gridworld" environments provide a tractable way for identifying pitfalls in the learned action policy. Indeed, in these simple environments two state-of-the-art DRL networks failed to effectively adapt to subtle differences between training and testing environments, highlighting the need for more robust RL and DRL algorithms. Another important environmental characteristic for the purposes of building model-based agents is the ability to alter, remove, or introduce dimensions of the environment without rendering previous training irrelevant. In other words, in order to fairly evaluate the success of the agent, there needs to exist a reliable strategy or internal model that is robust across variations of the environment or task rules [3]}.
