The goal of lifelong learning in AI is to be able to retain knowledge when trained on sequentially new tasks, and selectively transfer prior knowledge to learning the new tasks [1]}. For ANNs, methods for lifelong learning when encountering new tasks vary between constraining network weights using regularization, dynamically expanding capacity of the network while freezing the old capacity, or keeping a buffer of data from previous tasks to mix into data from the new tasks [2]}. The NCA keeps a buffer of data from previous experience for training the DNC model. In the context of RL, notable work on lifelong learning includes gradient-based meta-learning algorithms for fast and efficient adaption to new tasks [3]}, maintaining a mixture of models that are instantiated and recalled as new or old tasks are encountered [4]}, or learning local models for new tasks that can be bootstrapped with a globally maintained model [5]}. Our approach is closest to [5]} in learning a global model, but when encountering new tasks we leave the adaptation entirely unsupervised to the DNC, allowing it to leverage its algorithmic capabilities to transfer knowledge between multiple tasks.
