Continual learning aims to learn new tasks without forgetting previously learned ones. This is especially challenging when one cannot access data from previous tasks and when the model has a fixed capacity. Current regularization-based continual learning algorithms  need an external representation and extra computation to measure the parameters' \textit{importance}. In contrast, we propose Uncertainty-guided Continual Bayesian Neural Networks (UCB) where the learning rate adapts according to the uncertainty defined in the probability distribution of the weights in  networks. Uncertainty is a natural way to identify \textit{what to remember} and \textit{what to change} as we continually learn,  and thus mitigate catastrophic forgetting. We also show a variant of our model, which uses uncertainty for weight pruning 
 and retains task performance after pruning by saving binary masks per tasks. We evaluate our UCB approach extensively on diverse object classification datasets with short and long sequences of tasks and report superior or on-par performance compared to existing approaches. Additionally, we show that our model does not necessarily need task information at test time, i.e.~it does not presume knowledge of which task a sample belongs to. Humans can easily accumulate and maintain knowledge gained from previously observed tasks, and continuously learn to solve new problems or tasks. Artificial learning systems typically forget prior tasks when they cannot access all training data at once but are presented with task data in sequence. Overcoming these challenges is the focus of continual learning, sometimes also referred to as lifelong learning or sequential learning. Catastrophic forgetting (McCloskey & Cohen, 1989; McClelland et al., 1995) refers to the significant drop in the performance of a learner when switching from a trained task to a new one. This phenomenon occurs because trained parameters on the initial task change in favor of learning new objectives. This is the reason that naive finetuning intuitively suffers from catastrophic forgetting. Given a network of limited capacity, one way to address this problem is to identify the importance of each parameter and penalize further changes to those parameters that were deemed to be important for the previous tasks Aljundi et al., 2018a; Zenke et al., 2017 ). An alternative is to freeze the most important parameters and allow future tasks to only adapt the remaining parameters to new tasks (Mallya & Lazebnik, 2018) . Such models rely on the explicit parametrization of importance. We propose here implicit uncertainty-guided importance representation. Bayesian approaches to neural networks (MacKay, 1992b) can potentially avoid some of the pitfalls of explicit parameterization of importance in regular neural networks. Bayesian techniques, naturally account for uncertainty in parameters estimates. These networks represent each parameter with a distribution defined by a mean and variance over possible values drawn from a shared latent probability distribution (Blundell et al., 2015) . Variational inference can approximate posterior distributions using Monte Carlo sampling for gradient estimation. These networks act like ensemble methods in that they reduce the prediction variance but only use twice the number of parameters present in a regular neural network. We propose the use of the predicted mean and variance of the latent distributions to characterize the importance of each parameter. We perform continual learning with Bayesian neural networks by controlling the learning rate of each parameter as a function of its uncertainty. Figure 1 illustrates how posterior distributions evolve for certain and uncertain weight Illustration of evolution of weight distributions through learning two tasks. (a) circles represent weight parameters, initialized by distributions with mean and variance values randomly sampled from Ɲ(0,0.1). As an example we show five color-coded and plot their distributions. (b) Shows posterior distribution after learning Task 1. While W1 and W2 exhibit lower uncertainties (more contributions in learning Task 1), W3, W4, and W5 appear to have larger uncertainties, with the highest STD in W5, making them available to learn more tasks. (c) Task 2 is learned using higher learning rates for previously uncertain parameters (W3 and W4, W5) while learning rates for W1 and W2 are moderated according to their predicted low uncertainty after finishing task 1. Figure 1: Illustration of the evolution of weight distributions -uncertain weights adapt more quicklywhen learning two tasks using UCB. (a) weight parameter initialized by distributions initialized with mean and variance values randomly sampled from N (0, 0.1). (b) posterior distribution after learning task 1; while θ 1 and θ 2 exhibit lower uncertainties after learning the first task, θ 3 , θ 4 , and θ 5 have larger uncertainties, making them available to learn more tasks. (c) a second task is learned using higher learning rates for previously uncertain parameters (θ 1 , θ 2 , θ 3 , and θ 4 ) while learning rates for θ 1 and θ 2 are reduced. Size of the arrows indicate the magnitude of the change of the distribution mean upon gradient update. distributions while learning two consecutive tasks. Intuitively, the more uncertain a parameter is, the more learnable it can be and therefore, larger gradient steps can be taken for it to learn the current task. As a hard version of this regularization technique, we also show that pruning, i.e., preventing the most important model parameters from any change and learning new tasks with the remaining parameters, can be also integrated into UCB. We refer to this method as UCB-P. In this work, we propose a continual learning formulation with Bayesian neural networks, called UCB, that uses uncertainty predictions to perform continual learning: important parameters can be either fully preserved through a saved binary mask (UCB-P) or allowed to change conditioned on their uncertainty for learning new tasks (UCB). We demonstrated how the probabilistic uncertainty distributions per weight are helpful to continually learning short and long sequences of benchmark datasets compared against baselines and prior work. We show that UCB performs superior or on par with state-of-the-art models such as HAT across all the experiments. Choosing between the two UCB variants depends on the application scenario: While UCB-P enforces no forgetting after the initial pruning stage by saving a small binary mask per task, UCB does not require additional memory and allows for more learning flexibility in the network by allowing small forgetting to occur. UCB can also be used in a single head setting where the right subset of classes belonging to the task is not known during inference leading to a competitive model that can be deployed where it is not possible to distinguish tasks in a continuous stream of the data at test time. UCB can also be deployed in a single head scenario and where tasks information is not available at test time. A APPENDIX A.1 DATASETS Table 4 shows a summary of the datasets utilized in our work along with their size and number of classes. In all the experiments we resized images to 32 × 32 × 3 if necessary. For datasets with monochromatic images, we replicate the image across all RGB channels. (LeCun et al., 1998) 10 60,000 10,000 CIFAR100 (Krizhevsky & Hinton, 2009 ) 100 50,000 10,000 NotMNIST (Bulatov, 2011) 10 16,853 1,873 SVHN (Netzer et al., 2011) 10 73,257 26,032 CIFAR10 (Krizhevsky & Hinton, 2009) 10 39,209 12,630 TrafficSigns (Stallkamp et al., 2011) 43 39,209 12,630 FashionMNIST (Xiao et al., 2017) 10 60,000 10,000
A regularization-based approach for continual learning using Bayesian neural networks to predict parameters' importance
{"src": "scitldr-AIC_train", "paper_id": "HklUCCVKDB"}