Catastrophic forgetting in neural networks is one of the most well-known problems in continual learning. Previous attempts on addressing the problem focus on preventing important weights from changing. Such methods often require task boundaries to learn effectively and do not support backward transfer learning. In this paper, we propose a meta-learning algorithm which learns to reconstruct the gradients of old tasks w.r.t. the current parameters and combines these reconstructed gradients with the current gradient to enable continual learning and backward transfer learning from the current task to previous tasks. Experiments on standard continual learning benchmarks show that our algorithm can effectively prevent catastrophic forgetting and supports backward transfer learning.
 The ability to learn continually without forgetting previously learned skills is crucial to artificial general intelligence (AGI) BID3 . Addressing catastrophic forgetting in artificial neural networks (ANNs) has been the top priority of continual learning research. Notable attempts on solving the problem include Elastic Weight Consolidation (EWC) by BID2 and the follow up work on Synaptic Intelligence (SI) by BID6 , and Memory Aware Synapse (MAS) by BID0 . These algorithms share the same core idea: preventing important parameters from deviating from their old (presumably better) values. In order to achieve that, EWC-like algorithms compute the importance of each parameter w.r.t. each task in the sequence and for each old task, a regularization term is added to the loss of the new task to prevent that task from being catastrophically forgotten. The regular-Preliminary work. Under review by the International Conference on Machine Learning (ICML). Do not distribute. ization term for task T (i) in EWC-like algorithms takes the following form: DISPLAYFORM0 where λ (i) controls the relative importance of task i to the current task, θ is the current parameters, θ (i) * is the parameters found at the end of the training of T (i) , and ω DISPLAYFORM1 j is the importance of parameter θ 1. The regularizer in Eqn. 1 prevent changes to important parameters regardless of the effect of these changes. Unless θ DISPLAYFORM2 is the optimal value for the j-th parameter, either increasing or decreasing its value will result in better performance on task i. Keeping θ close to θ (i) * only prevent the network from catastrophically forgetting T (i) but cannot help the network to leverage the information from the current task T (k) , k > i to improve its performance on T (i) and other previous tasks. In other words, regularizers of the form in Eqn. 1 do not support backward transfer learning.2. The number of old parameter and importance vectors, θ * and ω, grows linearly with the number of tasks, making EWC-like algorithms not scalable to a large number of tasks. BID5 proposed the online EWC algorithm which maintains only one copy of θ * and ω. The sizes of θ * and ω are equal to that of the network. Therefore, the memory requirement of online EWC is still considerably large for large networks.To address these limitations of EWC-like algorithms, we propose a meta learning algorithm which:1. Learns to approximate the gradient of a task w.r.t. the current parameters from the current parameters 2. Combines the approximated gradients of old tasks w.r.t. the current parameters and the current task's gradient to result in an update that improves the performance of the network on all tasks.By combining the gradients, our algorithm exploits the similarity between the current task and previous tasks to enable backward transfer learning. As described in section 2.2 and 5.2, the size of a meta-network is typically orders of magnitude smaller than that of the main network and metanetworks for different tasks can be distilled into a single meta-network in an online manner. That significantly reduces the memory requirement of our method.In the next section, we introduce our learning to learn algorithm for continual learning. Experiments are presented in section 3. Conclusions and future work are located in section 4 and 5, respectively. In this paper, we present a meta learning algorithm for continual learning. Experiments on Permuted MNIST dataset show that our algorithm is effective in preventing catastrophic forgetting and is capable of supporting backward transfer learning.
We propose a meta learning algorithm for continual learning which can effectively prevent catastrophic forgetting problem and support backward transfer learning.
{"src": "scitldr-AIC_train", "paper_id": "SJgMHcrs3E"}