We present PROPS, a lightweight transfer learning mechanism for sequential data. PROPS learns probabilistic perturbations around the predictions of one or more arbitrarily complex, pre-trained black box models (such as recurrent neural networks). The technique pins the black-box prediction functions to "source states" of a hidden Markov model (HMM), and uses the remaining states as "perturbation states" for learning customized perturbations around those predictions. In this paper, we describe the PROPS model, provide an algorithm for online learning of its parameters, and demonstrate the consistency of this estimation. We also explore the utility of PROPS in the context of personalized language modeling. In particular, we construct a baseline language model by training a LSTM on the entire Wikipedia corpus of 2.5 million articles (around 6.6 billion words), and then use PROPS to provide lightweight customization into a personalized language model of President Donald J. Trump’s tweeting. We achieved good customization after only 2,000 additional words, and find that the PROPS model, being fully probabilistic, provides insight into when President Trump’s speech departs from generic patterns in the Wikipedia corpus. All code (both the PROPS training algorithm as well as reproducible experiments) are available as a pip-installable Python package.1
The (RNN-based) language model personalization scheme of @cite_6 provides many of the desiderata outlined in the introduction. By forming a source RNN on a large dataset and then retraining only the last layer on the endpoint, they obtain a lightweight, fast transfer learning scheme for sequential data that respects user privacy. The primary drawback of @cite_6 relative to is the loss of interpretability in the personalized model. Because the hidden variables of that model are not stochastic, one loses any insight into the respective contributions of the original source model vs. the personalization modes towards predictions about upcoming behavior. A subsidiary drawback is the loss of reliability as the more expressive model will also have higher variance.
@cite_6
2574917025
In this paper, we propose an efficient transfer leaning methods for training a personalized language model using a recurrent neural network with long short-term memory architecture. With our proposed fast transfer learning schemes, a general language model is updated to a personalized language model with a small amount of user data and a limited computing resource. These methods are especially useful for a mobile device environment while the data is prevented from transferring out of the device for privacy purposes. Through experiments on dialogue data in a drama, it is verified that our transfer learning methods have successfully generated the personalized language model, whose output is more similar to the personal language style in both qualitative and quantitative aspects.