Lifelong machine learning focuses on adapting to novel tasks without forgetting the old tasks, whereas few-shot learning strives to learn a single task given a small amount of data. These two different research areas are crucial for artificial general intelligence, however, their existing studies have somehow assumed some impractical settings when training the models. For lifelong learning, the nature (or the quantity) of incoming tasks during inference time is assumed to be known at training time. As for few-shot learning, it is commonly assumed that a large number of tasks is available during training. Humans, on the other hand, can perform these learning tasks without regard to the aforementioned assumptions. Inspired by how the human brain works, we propose a novel model, called the Slow Thinking to Learn (STL), that makes sophisticated (and slightly slower) predictions by iteratively considering interactions between current and previously seen tasks at runtime. Having conducted experiments, the results empirically demonstrate the effectiveness of STL for more realistic lifelong and few-shot learning settings. Deep Learning has been successful in various applications. However, it still has a lot of areas to improve on to reach human's lifelong learning ability. As one of its drawbacks, neural networks (NNs) need to be trained on large datasets before giving satisfactory performance. Additionally, they usually suffer from the problem of catastrophic forgetting (McCloskey & Cohen (1989); French (1999) )-a neural network performs poorly on old tasks after learning a novel task. In contrast, humans are able to incorporate new knowledge even from few examples, and continually throughout much of their lifetime. To bridge this gap between machine and human abilities, effort has been made to study few-shot learning (Fei-Fei et al. (2006) ; Lake et al. (2011); Santoro et al. (2016) ; Vinyals et al. (2016) ; Snell et al. (2017) ; Ravi & Larochelle (2017b) ; Finn et al. (2017) ; ; Garcia & Bruna (2018) ; Qi et al. (2018) ), lifelong learning (Gepperth & Karaoguz (2016) ; Rusu et al. (2016) ; Kirkpatrick et al. (2017) ; Yoon et al. (2018) ; ; ; SerrÃ et al. (2018) ; Schwarz et al. (2018) ; Sprechmann et al. (2018) ; Riemer et al. (2019) ), and both (Kaiser et al. (2017) ). The learning tasks performed by humans are, however, more complicated than the settings used by existing lifelong and few-shot learning works. Task uncertainty: currently, lifelong learning models are usually trained with hyperparameters (e.g., number of model weights) optimized for a sequence of tasks arriving at test time. The knowledge about future tasks (even their quantity) may be a too strong assumption in many real-world applications, yet without this knowledge, it is hard to decide the appropriate model architecture and capacity when training the models. Sequential few-shot tasks: existing few-shot learning models are usually (meta-)trained using a large collection of tasks. 1 Unfortunately, this collection is not available in the lifelong learning scenarios where tasks come in sequentially. Without seeing many tasks at training time, it is hard for an existing few-shot model to learn the shared knowledge behind the tasks and use the knowledge to speed up the learning of a novel task at test time. Humans, on the other hand, are capable of learning well despite having only limited information and/or even when not purposely preparing for a particular set of future tasks. Comparing how humans learn and think to how the current machine learning models are trained to learn and make predictions, we observe that the key difference lies on the part of thinking, which is the decision-making counterpart of models when making predictions. While most NN-based supervised learning models use a single forward pass to predict, humans make careful and less error-prone decisions in a more sophisticated manner. Studies in biology, psychology, and economics (Parisi et al. (2019) ; Kahneman & Egan (2011) ) have shown that, while humans make fast predictions (like machines) when dealing with daily familiar tasks, they tend to rely on a slow-thinking system that deliberately and iteratively considers interactions between current and previously learned knowledge in order to make correct decisions when facing unfamiliar or uncertain tasks. We hypothesize that this slow, effortful, and less error-prone decision-making process can help bridge the gap of learning abilities between humans and machines. We propose a novel brain-inspired model, called the Slow Thinking to Learn (STL), for taskuncertain lifelong and sequential few-shot machine learning tasks. STL has two specialized but dependent modules, the cross-task Slow Predictor (SP) and per-task Fast Learners (FLs), that output lifelong and few-shot predictions, respectively. We show that, by making the prediction process of SP more sophisticated (and slightly slower) at runtime, the learning process of all modules can be made easy at training time, eliminating the need to fulfill the aforementioned impractical settings. Note that the techniques for slow predictions (Finn et al. (2017) ; Ravi & Larochelle (2017b) ; Nichol & Schulman (2018) ; Sprechmann et al. (2018) ) and fast learning (McClelland et al. (1995) ; Kumaran et al. (2016) ; Kaiser et al. (2017) ) have already been proposed in the literature. Our contributions lie in that we 1) explicitly model and study the interactions between these two techniques, and 2) demonstrate, for the first time, how such interactions can greatly improve machine capability to solve the joint lifelong and few-shot learning problems encountered by humans everyday. 2 Slow Thinking to Learn (STL) Figure 1: The Slow Thinking to Learn (STL) model. To model the interactions between the shared SP f and per-task FLs {(g (t) , M (t) )} t , we feed the output of FLs into the SP while simultaneously letting the FLs learn from the feedback given by SP. We focus on a practical lifelong and fewshot learning set-up: , · · · arriving in sequence and the labeled examples also coming in sequence, the goal is to design a model such that it can be properly trained by data ) collected up to any given time point s, and then make correct predictions for unlabeled data X (t) = {x (t,i) } i in any of the seen tasks, t ≤ s. Note that, at training time s, the future tasks To solve Problem 1, we propose the Slow Thinking to Learn (STL) model, whose architecture is shown in Figure 1 . The STL is a cascade where the shared Slow Predictor (SP) network f parameterized by θ takes the output of multiple task-specific Fast Learners (FLs) {(g (t) , M (t) )} t , t ≤ s, as input. An FL for task T (t) consists of an embedding network g (t)2 parameterized by φ (t) and augmented with an external, episodic, non-parametric memory Here, we use the Memory Module (Kaiser et al. (2017) ) as the external memory which saves the clusters of seen examples {(x (t,i) , y (t,i) )} i to achieve better storage efficiency-the h (t,j) of an entry (h (t,j) , v (t,j) ) denotes the embedding of a cluster of x (t,i) 's with the same label while the v (t,j) denotes the shared label. We use the FL (g (t) , M (t) ) and SP f to make few-shot and lifelong predictions for task T (t) , respectively. We let the number of FLs grow with the number of seen tasks in order to ensure that the entire STL model will have enough complexity to learn from possibly endless tasks in lifelong. This does not imply that the SP will consume unbounded memory space to make predictions at runtime, as the FL for a specific task can be stored on a hard disk and loaded into the main memory only when necessary. Slow Predictions. The FL predicts the label of a test instance x using a single feedforward pass just like most existing machine learning models. As shown in Figure 2 (a), the FL for task T (t) first embed the instance to get h = g (t) (x ) and then predicts the labelŷ FL of x by averaging the cluster labels where KNN(h ) is the set of K nearest neighboring embeddings of h . We havê where h, h denotes the cosine similarity between h (t,j) and h . On the other hand, the SP predicts the label of x with a slower, iterative process, which is shown in Figure 2 (b). The SP first adapts (i.e., fine-tunes) its weights θ to KNN(h ) and their corresponding values stored in M (t) to getθ by solving where loss(·) denotes a loss function. Then, the SP makes a prediction byŷ SP = f (h ;θ ). The adapted network fθ is discarded after making the prediction. The slower decision-making process of SP may seem unnecessary and wasteful of computing resources at first glance. Next, we explain why it is actually a good bargain. Life-Long Learning with Task Uncertainty. Since the SP makes predictions after runtime adaptation, we define the training objective of θ for task T (s) such that it minimizes the losses after being adapted for each seen task The term loss(f (h;θ * ), v) denotes the empirical slow-prediction loss of the adapted SP on an example (x, y) in M (t) , whereθ * denotes the weights of the adapted SP for x following Eq. (1): requires recursively solvingθ * for each (x, y) remembered by the FLs. We use an efficient gradient-based approach proposed by Finn et al. (2017) ) to solve Eq. (2). Please refer to Section 2.1 of the Appendix for more details. Since the SP learns from the output of FLs, theθ * in Eq. (2) approximates a hypothesis used by an FL to predict the label of x. The θ, after being trained, will be close to everyθ * and can be fine-tuned to become a hypothesis, meaning that θ encodes the invariant principles 3 underlying the hypotheses for different tasks. (a) (b) (c) Figure 3 : The relative positions between the invariant representations θ and the approximate hypothesesθ (t) 's of FLs for different tasks T (t) 's on the loss surface defined by FLs after seeing the (a) first, (b) second, and (c) third task. Since θ−θ (t) ≤ R for any t in Eq. (2), the effective capacity of SP (at runtime) is the union of the capacity of all possible points within the dashed R-circle centered at θ. Furthermore, after being sequentially trained by two tasks using Eq. (3), the θ will easily get stuck in the middle ofθ (1) andθ (2) . To solve the third task, the third FL needs to change its embedding function (and therefore the loss surface) such thatθ (3) falls into the R-circle centered at θ. Recall that in Problem 1, the nature of tasks arriving after a training process is unknown, thus, it is hard to decide the right model capacity at training time. A solution to this problem is to use an expandable network (Rusu et al. (2016) ; Yoon et al. (2018) ) and expand the network when training it for a new task, but the number of units to add during each expansion remains unclear. Our STL walks around this problem by not letting the SP learn the tasks directly but making it learn the invariant principles behind the tasks. Assuming that the underlying principles of the learned hypotheses for different tasks are universal and relatively simple, 4 one only needs to choose a model architecture with capacity that is enough to learn the shared principles in lifelong manner. Note that limiting the capacity of SP at training time does not imply underfitting. As shown in Figure 3 , the postadaptation capacity of SP at runtime can be much larger than the capacity decided during training. Sequential Few-Shot Learning. Although each FL is augmented with an external memory that has been shown to improve learning efficiency by the theory of complementary learning systems (McClelland et al. (1995) ; Kumaran et al. (2016) ), it is not sufficient for FLs to perform few-shot predictions. Normally, these models need to be trained on many existing few-shot tasks in order to obtain good performance at test time. Without assuming s in Problem 1 to be a large number, the STL takes a different approach that fast stabilizes θ and then let the FL for a new incoming task learn a good hypothesis by extrapolating from θ. We define the training objective of g (s) , which is parameterized by φ (s) and augmented with memory M (s) , for the current task T (s) as follows: where ) is the empirical loss term whose specific form depends on the type of external memory used (see Section 2.2 of the Appendix for more details), and ) is a regularization term, which we call the feedback term, whose inverse value denotes the usefulness of the FL in helping SP (f parameterized by θ) adapt. Specifically, it is written as The feedback term encourages each FL to learn unique and salient features for the respective task so the SP will not be confused by two tasks having similar embeddings. As shown in Figure 3 (b), the relative position of θ gets "stuck" easily after seeing a few of previous tasks. To solve the current task, g (s) needs to change the loss surface for θ such thatθ (s) falls into the R-circle centered at θ (Figure 3(c) ). This makes θ an efficient guide (through the feedback term) to finding g (s) when there are only few examples and also few previous tasks. We use an alternate training procedure to train the SP and FLs. Please see Section 2.3 of the Appendix for more details. Note that when sequentially training STL for task T (s) in lifelong, we can safely discard the data in the previous tasks because the FLs are task-specific (see Eq. (3)) and the SP does not require raw examples to train (see Eq. (2)). Inspired by the thinking process that humans undergo when making decisions, we propose STL, a cascade of per-task FLs and shared SP. To the best of our knowledge, this is the first work that studies the interactions between the fast-learning and slow-prediction techniques and shows how such interactions can greatly improve machine capability to solve the joint lifelong and few-shot learning problems under challenging settings. For future works, we will focus on integrating the STL with different types of external memory and studying the performance of STL in real-world deployments.
This paper studies the interactions between the fast-learning and slow-prediction models and demonstrate how such interactions can improve machine capability to solve the joint lifelong and few-shot learning problems.
{"src": "scitldr-AIC_train", "paper_id": "HklbIerFDS"}