We present PROPS, a lightweight transfer learning mechanism for sequential data. PROPS learns probabilistic perturbations around the predictions of one or more arbitrarily complex, pre-trained black box models (such as recurrent neural networks). The technique pins the black-box prediction functions to "source states" of a hidden Markov model (HMM), and uses the remaining states as "perturbation states" for learning customized perturbations around those predictions. In this paper, we describe the PROPS model, provide an algorithm for online learning of its parameters, and demonstrate the consistency of this estimation. We also explore the utility of PROPS in the context of personalized language modeling. In particular, we construct a baseline language model by training a LSTM on the entire Wikipedia corpus of 2.5 million articles (around 6.6 billion words), and then use PROPS to provide lightweight customization into a personalized language model of President Donald J. Trump’s tweeting. We achieved good customization after only 2,000 additional words, and find that the PROPS model, being fully probabilistic, provides insight into when President Trump’s speech departs from generic patterns in the Wikipedia corpus. All code (both the PROPS training algorithm as well as reproducible experiments) are available as a pip-installable Python package.1
A (SRNN) @cite_2 addresses a shortcoming of RNN's relative to state-space models such as HMM's by allowing for stochastic hidden variables. We surmise that the the SRNN framework will eventually generate a state-of-the-art transfer learning mechanism for sequential data that satisfies the interpretability desideratum from the introduction. However, to our knowledge, such a mechanism has not yet been developed. Moreover, the training of a SRNN is substantially more complex than training a standard RNN, let alone a HMM, and one would expect that computational complexity to spill over into the transference algorithm. If so, would provide a lightweight alternative.
@cite_2
2963279312
How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The clear separation of deterministic and stochastic layers allows a structured variational inference network to track the factorization of the model’s posterior distribution. By retaining both the nonlinear recursive structure of a recurrent neural network and averaging over the uncertainty in a latent path, like a state space model, we improve the state of the art results on the Blizzard and TIMIT speech modeling data sets by a large margin, while achieving comparable performances to competing methods on polyphonic music modeling.