One of the key differences between the learning mechanism of humans and Artificial Neural Networks (ANNs) is the ability of humans to learn one task at a time. ANNs, on the other hand, can only learn multiple tasks simultaneously. Any attempts at learning new tasks incrementally cause them to completely forget about previous tasks. This lack of ability to learn incrementally, called Catastrophic Forgetting, is considered a major hurdle in building a true AI system. In this paper, our goal is to isolate the truly effective existing ideas for incremental learning from those that only work under certain conditions. To this end, we first thoroughly analyze the current state of the art (iCaRL) method for incremental learning and demonstrate that the good performance of the system is not because of the reasons presented in the existing literature. We conclude that the success of iCaRL is primarily due to knowledge distillation and recognize a key limitation of knowledge distillation, i.e, it often leads to bias in classifiers. Finally, we propose a dynamic threshold moving algorithm that is able to successfully remove this bias. We demonstrate the effectiveness of our algorithm on CIFAR100 and MNIST datasets showing near-optimal results. Our implementation is available at this https URL.
Catastrophic forgetting occurs when connectionist networks learn new information, and by so doing, forget all previously learned information. This workshop focused primarily on the causes of catastrophic interference, the techniques that have been developed to reduce it, the effect of these techniques on the networks' ability to generalize, and the degree to which prediction of catastrophic forgetting is possible. The speakers were Robert French, Phil Hetherington (Psychology Department, McGill University, het@blaise.psych.mcgill.ca), and Stephan Lewandowsky (Psychology Department, University of Oklahoma, lewan@constellation.ecn.uoknor.edu). Resume Les procedures d'apprentissage de descente de gradient sont le plus souvent utilisees dans la modelisation de reseaux neuronaux. Lorsque ces algorithmes (e.g., la retropropagation) sont appliques a des tâches d'apprentissage sequentiel, un defaut majeur, appele oubli catastrophique (ou interference catastrophique), survient generalement: lorsqu'un reseau ayant deja appris un premier ensemble d'items est ensuite entraine sur un deuxieme ensemble d'items, alors l'information nouvellement apprise peut completement detruire celle anterieurement apprise. Pour eviter ce defaut peu plausible, il est ici propose une architecture a deux reseaux neuronaux ou les nouveaux items sont appris par un premier reseau conjointement avec des pseudo-items internes provenant d'un second reseau. Comme il est demontre que ces pseudo-items refletent la structure des items anterieurement appris par le premier reseau, le modele implemente ainsi un processus de rafraichissement par l'ancienne information. Le point crucial est que ce mecanisme de rafraichissement repose sur des reseaux neuronaux reverberants qui n'ont besoin que de stimulations aleatoires pour operer. Le modele fournit ainsi un moyen de reduire d'une maniere spectaculaire l'interference retroactive tout en conservant la nature essentiellement distribuee de l'information et propose une hypothese originale et plausible sur la maniere de ≪ copier et coller ≫ la memoire distribuee au sein d'une structure cerebrale vers une autre. Catastrophic forgetting is a problem faced by many machine learning models and algorithms. When trained on one task, then trained on a second task, many machine learning models "forget" how to perform the first task. This is widely believed to be a serious problem for neural networks. Here, we investigate the extent to which the catastrophic forgetting problem occurs for modern neural networks, comparing both established and recent gradient-based training algorithms and activation functions. We also examine the effect of the relationship between the first task and the second task on catastrophic forgetting. We find that it is always best to train using the dropout algorithm--the dropout algorithm is consistently best at adapting to the new task, remembering the old task, and has the best tradeoff curve between these two extremes. We find that different tasks and relationships between tasks result in very different rankings of activation function performance. This suggests the choice of activation function should always be cross-validated. Publisher Summary Connectionist networks in which information is stored in weights on connections among simple processing units have attracted considerable interest in cognitive science. Much of the interest centers around two characteristics of these networks. First, the weights on connections between units need not be prewired by the model builder but rather may be established through training in which items to be learned are presented repeatedly to the network and the connection weights are adjusted in small increments according to a learning algorithm. Second, the networks may represent information in a distributed fashion. This chapter discusses the catastrophic interference in connectionist networks. Distributed representations established through the application of learning algorithms have several properties that are claimed to be desirable from the standpoint of modeling human cognition. These properties include content-addressable memory and so-called automatic generalization in which a network trained on a set of items responds correctly to other untrained items within the same domain. New learning may interfere catastrophically with old learning when networks are trained sequentially. The analysis of the causes of interference implies that at least some interference will occur whenever new learning may alter weights involved in representing old learning, and the simulation results demonstrate only that interference is catastrophic in some specific networks. Abstract All natural cognitive systems, and, in particular, our own, gradually forget previously learned information. Plausible models of human cognition should therefore exhibit similar patterns of gradual forgetting of old information as new information is acquired. Only rarely does new learning in natural cognitive systems completely disrupt or erase previously learned information; that is, natural cognitive systems do not, in general, forget ‘catastrophically'. Unfortunately, though, catastrophic forgetting does occur under certain circumstances in distributed connectionist networks. The very features that give these networks their remarkable abilities to generalize, to function in the presence of degraded input, and so on, are found to be the root cause of catastrophic forgetting. The challenge in this field is to discover how to keep the advantages of distributed connectionist networks while avoiding the problem of catastrophic forgetting. In this article the causes, consequences and numerous solutions to the problem of catastrophic forgetting in neural networks are examined. The review will consider how the brain might have overcome this problem and will also explore the consequences of this solution for distributed connectionist networks.