Most research on lifelong learning applies to images or games, but not language.
 We present LAMOL, a simple yet effective method for lifelong language learning (LLL) based on language modeling.
 LAMOL replays pseudo-samples of previous tasks while requiring no extra memory or model capacity.
 Specifically, LAMOL is a language model that simultaneously learns to solve the tasks and generate training samples.
 When the model is trained for a new task, it generates pseudo-samples of previous tasks for training alongside data for the new task.
 The results show that LAMOL prevents catastrophic forgetting without any sign of intransigence and can perform five very different language tasks sequentially with only one model. 
 Overall, LAMOL outperforms previous methods by a considerable margin and is only 2-3% worse than multitasking, which is usually considered the LLL upper bound.
 The source code is available at https://github.com/jojotenya/LAMOL. The current dominant paradigm for machine learning is to run an algorithm on a given dataset to produce a trained model specifically for a particular purpose; this is isolated learning (Chen & Liu, 2016, p. 150) . In isolated learning, the model is unable to retain and accumulate the knowledge it has learned before. When a stream of tasks are joined to be trained sequentially, isolated learning faces catastrophic forgetting (McCloskey & Cohen, 1989) due to a non-stationary data distribution that biases the model (left figure of Figure 1 ). In contrast, lifelong learning is designed to address a stream of tasks by accumulating interconnected knowledge between learned tasks and retaining the performance of those tasks. A human easily achieves lifelong learning, but this is nontrivial for a machine; thus lifelong learning is a vital step toward artificial general intelligence. In this paper, we focus on lifelong language learning, where a machine achieves lifelong learning on a stream of natural language processing (NLP) tasks. To the best of our knowledge, lifelong language learning has been studied in only a few instances; for sentiment analysis (Chen et al., 2015b; Xia et al., 2017) , conversational agents (Lee, 2017) , word representation learning (Xu et al., 2018) , sentence representation learning (Liu et al., 2019 ), text classification, and question answering (d'Autume et al., 2019) . However, in all previous work, the tasks in the stream are essentially the same task but in different domains. To achieve lifelong language learning on fundamentally different tasks, we propose LAMOL -LAnguage MOdeling for Lifelong language learning. It has been shown that many NLP tasks can be considered question answering (QA) (Bryan McCann & Socher, 2018) . Therefore, we address multiple NLP tasks with a single model by training a language model (LM) that generates an answer based on the context and the question. Treating QA as language modeling is beneficial because the LM can be pre-trained on a large number of sentences without any labeling (Radford et al., 2019) ; however, this does not directly solve the problem of LLL. If we train an LM on a stream of tasks, catastrophic forgetting still occurs. However, as an LM is intrinsically a text generator, we can use it to answer questions while generating pseudo-samples of Figure 1 : Left: After learning Task 2, the learner has already forgetten how to solve Task 1. This is "catastrophic forgetting". Middle: The basic idea of the data-based LLL approach. A generator is learned to generate examples it has seen before. Using the generator, the learner also learns from examples from the previous task to prevent it from forgetting. Right: A language model that simultaneously takes on the roles of learner and generator. the previous task to be replayed later. LAMOL is inspired by the data-based approach for LLL in which a generator learns to generate samples in previous tasks (middle of Figure 1 ) (Hanul Shin & Kim, 2017; Kemker & Kanan, 2017) . In contrast to previous approaches, LAMOL needs no extra generator (right of Figure 1 ). LAMOL is also similar to multitask training, but the model itself generates data from previous tasks instead of using real data. Our main contributions in this paper are: • We present LAMOL, a simple yet effective method for LLL. Our method has the advantages of no requirements in terms of extra memory or model capacity. We also do not need to know how many tasks to train in advance and can always train on additional tasks when needed. • Experimental results show that our methods outperform baselines and other state-of-the-art methods by a considerable margin and approaches the multitasking upper bound within 2-3%. • Furthermore, we propose adding task-specific tokens during pseudo-sample generation to evenly split the generated samples among all previous tasks. This extension stabilizes LLL and is particularly useful when training on a large number of tasks. • We analyze how different amounts of pseudo-samples affect the final performance of LAMOL, considering results both with and without the task-specific tokens. • We open-source our code to facilitate further LLL research. We propose LAMOL, a simple yet effective method for LLL based on language modeling. A single LM achieves LLL without additional model components and without keeping old examples. Moreover, any pre-trained LM can be used to leverage a large amount of unlabeled text to improve LLL. Finally, more tasks can be added whenever needed.
Language modeling for lifelong language learning.
{"src": "scitldr-AIC_train", "paper_id": "Skgxcn4YDS"}