The environments tested are simple for the purpose of establishing proof of concept in using a global DNC model in the continual RL setting. The environments represent multiple tasks requiring an RL agent and predictive model to learn continually. In the directional path navigation environment, the numbered tiles change for the same navigation task as the agent solves each level. In the obstacle-based grid navigation environment, new obstacles are presented, requiring the agent to adapt its navigation strategy. Other work on continual learning in the RL context examines environments where the tasks involve locomotion of agents represented as half-cheetahs, ants, or spiders [1]}, [2]}, [3]}. The tasks are adapted in a number of ways such as varying the nonstationary locomotion environment the agent is in (the background, the slope of the terrain, and so on), or crippling the agent by disabling a joint or a leg. The agent's continual learning capabilities are tested as the tasks are varied. An extension of our work would be to test the NCA on such environments.
