LSTM-based language models exhibit compositionality in their representations, but how this behavior emerges over the course of training has not been explored. Analyzing synthetic data experiments with contextual decomposition, we find that LSTMs learn long-range dependencies compositionally by building them from shorter constituents during training. Consider the process of backpropagation through time for a language model. As an example, the language model should learn that an occurrence of "either" increases the later likelihood of "or". To do so, it must backpropagate information from the occurrence of "or" through some intervening constituent, which we will refer to as a conduit because the association of either/or is carried through it to affect the representation of "either". Perhaps it encounters a training example that uses a conduit that is predictable by being structured in familiar ways, here italicized: "Either Socrates is mortal or not all men are mortal." However, what if the conduit is unpredictable and the structure cannot be interpreted by the model, for example, if the conduit includes unknown tokens, as in: "Either slithy toves gyre or mome raths outgrabe"? Which conduit will carry the gradient from "or" to "either" easily?Formally , as the gradient of the error e t at timestep t is backpropagated k timesteps through the hidden state h: DISPLAYFORM0 The backpropagated message is multiplied repeatedly by the gradients associated with each item in the conduit. If the recurrence derivatives ∂h i+1 ∂h i are large at some parameter, the correspondingly larger backpropagated gradient ∂et ∂h t−k will accelerate descent in that direction.When we ask which conduit will carry the gradient message to learn a long-range dependency faster, the answer will depend on the magnitude and distribution of the recurrence gradients. If the language model relies on linguistic structure in the conduit in order to pass the message effectively, then the more predictable conduit will facilitate learning a long-range pattern.In order to investigate whether long-range dependencies are built from short constituents, we train models on synthetic data which varies the predictability of short sequences. We find that memorizing local patterns allows a language model to learn a long-range dependency faster but ultimately inhibits its ability to fully acquire longrange rules. We confirm that the longer the span of a rule, the more examples are required for an LSTM model to effectively learn the rule. We then find1 that a more predictable conduit between the rule symbols promotes the early learning of the rule, implying that the process by which an LSTM learns long-range rules is compositional. However, the representation learned through the predictable conduit ultimately prevents the model from confidently learning these long-range connections.
LSTMs learn long-range dependencies compositionally by building them from shorter constituents over the course of training.
{"src": "scitldr-AIC_train", "paper_id": "BJl3s1h9aV"}