AI agents can be trained to perform tasks in simulations generated by a predictive model of the environment [1]}, [2]}, [3]}. Where such a model is learned using ANNs, we hypothesize that DNCs present a robust mechanism that is well suited for lifelong learning. The algorithmic capabilities of the DNC can be leveraged to simulate multiple tasks present in an environment, which can be useful for continual model-based Reinforcement Learning (RL). To test this, we devise an architecture for lifelong RL where an ANN agent and DNC model are trained in conjunction to iteratively learn to solve tasks in a given environment. We call our architecture the Neural Computer Agent (NCA). In the NCA, we use a Convolutional Autoencoder to learn low dimensional latent state representations from the frames, consisting of pixels, collected by the agent in the environment. The DNC model is trained to predict the environment states in this latent space, and the agent is trained to use the latent states and information from the model. We use the model to simulate the environment entirely in the latent space, where the agent is trained using RL. The agent then rolls out to the live environment to collect new experience which is used to further train the DNC model.
