This paper addresses the problem of incremental domain adaptation (IDA). We assume each domain comes sequentially, and that we could only access data in the current domain. The goal of IDA is  to build a unified model performing well on all the encountered domains. We propose to augment a recurrent neural network (RNN) with a directly parameterized memory bank, which is retrieved by an attention mechanism at each step of RNN transition. The memory bank provides a natural way of IDA: when adapting our model to a new domain, we progressively add new slots to the memory bank, which increases the model capacity. We learn the new memory slots and fine-tune existing parameters by back-propagation. 
 Experiments show that our approach significantly outperforms naive fine-tuning and previous work on IDA, including elastic weight consolidation and the progressive neural network.   Compared with expanding hidden states, our approach is more robust for old domains, shown by both empirical and theoretical results. Domain adaptation aims to transfer knowledge from a source domain to a target domain in a machine learning system. This is important for neural networks, which are data-hungry and prone to overfitting. In this paper, we focus on incremental domain adaptation (IDA), where we assume different domains come one after another. We only have access to the data in the current domain, but hope to build a unified model that performs well on all the domains that we have encountered (Xu et al., 2014; Rusu et al., 2016; Kirkpatrick et al., 2017) .Incremental domain adaptation is useful in various scenarios. Suppose a company is doing business with different partners over a long period of time. The company can only access the data of the partner with a current contract. However, the machine learning model is the company's property (if complying with the contract). Therefore, it is desired to preserve as much knowledge as possible in the model and not to rely on the availability of the data.Another important application of IDA is a quick adaptation to new domains. If the environment of a deployed machine learning system changes frequently, traditional methods like jointly training all domains require the learning machine to be re-trained from scratch every time. Fine-tuning a neural network by a few steps of gradient updates does transfer quickly, but it suffers from the catastrophic forgetting problem (Kirkpatrick et al., 2017) . Suppose we do not know the domain of a data point when predicting; the (single) finetuned model cannot predict well for samples in previous domains, as it tends to "forget" quickly during fine-tuning.A recent trend of domain adaptation in the deep learning regime is the progressive neural network (Rusu et al., 2016) , which progressively grows the network capacity if a new domain comes. Typically, this is done by enlarging the model with new hidden states and a new predictor ( FIG0 ). To avoid interfering with existing knowledge , the newly added hidden states are not fed back to the previously trained states. During training, they fix all existing parameters , and only train the newly added ones. For inference, they use the new predictor for all domains. This is sometimes undesired as the new predictor is trained with only the last domain.In this paper, we propose a progressive memory bank for incremental domain adaptation. Our model augments a recurrent neural network (RNN ) with a memory bank, which is a set of distributed, real-valued vectors capturing domain knowledge. The memory is retrieved by an attention mechanism . When our model is adapted to new domains, we progressively increase the slots in the memory bank. But different from (Rusu et al., 2016) , we fine-tune all the parameters, including RNN and the existing memory slots. Empirically, when the model capacity increases, the RNN does not forget much even if the entire network is fine-tuned. Compared with expanding RNN hidden states, the newly added memory slots do not contaminate existing knowledge in RNN states, as will be shown by a theorem. We evaluate our approach 1 on Natural Language Inference and Dialogue Response Generation. Experiments support our hypothesis that the proposed approach adapts well to target domains without catastrophic forgetting of the source. Our model outperforms the naïve fine-tuning method, the original progressive neural network, as well as other IDA techniques including elastic weight consolidation (EWC) (Kirkpatrick et al., 2017) .Detailed related work is provided in Appendix A. In this paper, we propose a progressive memory network for incremental domain adaptation (IDA). We augment an RNN with an attention-based memory bank. During IDA, we add new slots to the memory bank and tune all parameters by back-propagation. Empirically, the progressive memory network does not suffer from the catastrophic forgetting problem as in naïve fine-tuning. Our intuition is that the new memory slots increase the neural network's model capacity, and thus, the new knowledge less overrides the existing network. Compared with expanding hidden states, our progressive memory bank provides a more robust way of increasing model capacity, shown by both a theorem and experiments. We also outperform previous work for IDA, including elastic weight consolidation (EWC) and the original progressive neural network.
We present a neural memory-based architecture for incremental domain adaptation, and provide theoretical and empirical results.
{"src": "scitldr-AIC_train", "paper_id": "rJgkE5HsnV"}