Catastrophic forgetting poses a grand challenge for continual learning systems, which prevents neural networks from protecting old knowledge while learning new tasks sequentially. We propose a Differentiable Hebbian Plasticity (DHP) Softmax layer which adds a fast learning plastic component to the slow weights of the softmax output layer. The DHP Softmax behaves as a compressed episodic memory that reactivates existing memory traces, while creating new ones. We demonstrate the flexibility of our model by combining it with existing well-known consolidation methods to prevent catastrophic forgetting. We evaluate our approach on the Permuted MNIST and Split MNIST benchmarks, and introduce Imbalanced Permuted MNIST — a dataset that combines the challenges of class imbalance and concept drift. Our model requires no additional hyperparameters and outperforms comparable baselines by reducing forgetting. A key aspect of human intelligence is the ability to continually adapt and learn in dynamic environments, a characteristic which is challenging to embed into artificial intelligence. Recent advances in machine learning (ML) have shown tremendous improvements in various problems, by learning to solve one complex task very well, through extensive training on large datasets with millions of training examples or more. Most of the ML models that we use during deployment assume that the real-world is stationary, where in fact it is non-stationary and the distribution of acquired data changes over time. Therefore, after learning is complete, and these models are fine-tuned with new data, performance degrades with respect to the original data. This phenomenon *Work done during an internship at Uber AI. † Work done while at Google Brain. known as catastrophic forgetting or catastrophic interference BID17 BID7 serves to be a crucial problem for deep neural networks (DNNs) that are tasked with continual learning BID26 or lifelong learning (Thrun & Mitchell, 1995) . In this learning paradigm, the goal is to adapt and learn consecutive tasks without forgetting how to perform previously learned tasks. Some of the real-world applications that typically require this kind of learning include perception for autonomous vehicles, recommender systems, fraud detection, etc.In most supervised learning methods, DNN architectures require independent and identically distributed (iid) samples from a stationary training distribution. However, for ML systems that require continual learning in the real-world, the iid assumption is easily violated when: (1) There is concept drift or class imbalance in the training data distribution.(2 ) Data representing all scenarios in which the learner is expected to perform are not initially available. In such situations, DNNs face the "stability-plasticity dilemma" BID6 BID0 . This presents a continual learning challenge for models that need to balance plasticity (integrate new knowledge) and stability (preserve existing knowledge).Two major theories have been proposed to explain a human's ability to perform continual learning. The first theory is inspired by synaptic consolidation in the mammalian neocortex BID5 where a subset of synapses are rendered less plastic and therefore preserved for a longer timescale. The second theory is the complementary learning systems (CLS) theory BID16 BID23 BID12 , which suggests that humans extract high-level structural information and store it in a different brain area while retaining episodic memories.Here, we extend the work on differentiable plasticity BID18 BID19 to a continual learning setting and develop a model that is capable of adapting quickly to changing environments as well as consolidating previous knowledge by selectively adjusting the plasticity of synapses. We modify the traditional softmax layer and propose to augment the slow weights with a set of plastic weights implemented using Differentiable Hebbian Plasticity (DHP). The model 's slow weights learn deep representations of data and the fast weights implemented with DHP learn to quickly "auto-associate" the class labels to representations. We also demonstrate the flexibility of our model by combining it with recent task-specific synaptic consolidation based methods to overcoming catastrophic forgetting such as elastic weight consolidation BID11 BID28 , synaptic intelligence (Zenke et al., 2017) and memory aware synapses . Our model unifies core concepts from Hebbian plasticity, synaptic consolidation and CLS theory to enable rapid adaptation to new unseen data, while consolidating synapses and leveraging compressed episodic memories to remember previous knowledge and mitigate catastrophic forgetting. We have shown that the problem of catastrophic forgetting in continual learning environments can be alleviated by adding compressed episodic memory in the softmax layer through DHP. DHP Softmax alone showed noticeable improvement across all benchmarks when compared to a neural network with a traditional softmax layer. We demonstrated the flexibility of our model where, in addition to DHP Softmax, we can regularize the slow weights using EWC, SI or MAS to improve a model's ability to alleviate catastrophic forgetting. The approach where we combine DHP Softmax and MAS consistently leads to overall superior results compared to other baseline methods on several benchmarks. This gives a strong indication that Hebbian plasticity enables neural networks to learn continually and remember distant memories, thus reducing catastrophic forgetting when learning from sequential datasets in dynamic environments. For the Imbalanced Permuted MNIST experiments shown in Figure 2 , the regularization hyperparameter λ for each of the task-specific consolidation methods is λ = 400 for Online EWC BID28 , λ = 1.0 for SI (Zenke et al., 2017) and λ = 0.1 for MAS . In SI, the damping parameter, ξ, was set to 0.1. Similar to the Permuted MNIST benchmark, to find the best hyperparameter combination for each of these synaptic consolidation methods, we performed a grid search using a task sequence determined by a single seed. Across all experiments, we maintained the the same random probabilities detemined by a single seed to artificially remove training samples from each class. The hyperparameters of the synaptic consolidation methods (i.e. Online EWC, SI and MAS) remain the same with and without DHP Softmax, and the plastic components are not regularized.
Hebbian plastic weights can behave as a compressed episodic memory storage in neural networks; improving their ability to alleviate catastrophic forgetting in continual learning.
{"src": "scitldr-AIC_train", "paper_id": "r1x-E5Ss34"}