Recurrent  neural  networks  (RNNs)  are  a  powerful tool for modeling sequential data. Despite their widespread usage, understanding how RNNs solve complex problems remains elusive.   Here, we characterize how popular RNN architectures perform document-level sentiment classification. Despite their theoretical capacity to implement complex, high-dimensional computations, we find that  trained  networks  converge  to  highly  interpretable, low-dimensional representations.   We identify a simple mechanism, integration along an approximate line attractor, and find this mechanism present across RNN architectures (including LSTMs, GRUs, and vanilla RNNs). Overall, these results demonstrate that surprisingly universal and human interpretable computations can arise across a range of recurrent networks. Recurrent neural networks (RNNs) are a popular tool for sequence modelling tasks. These architectures are thought to learn complex relationships in input sequences, and exploit this structure in a nonlinear fashion. RNNs are typically viewed as black boxes, despite considerable interest in better understanding how they function.Here, we focus on studying how recurrent networks solve document-level sentiment analysis-a simple, but longstanding benchmark task for language modeling BID6 BID13 . We demonstrate that popular RNN architectures, despite having the capacity to implement high-dimensional and nonlinear computations, in practice converge to low-dimensional representations when trained against this task. Moreover, using analysis techniques from dynamical systems theory, we show that locally linear approximations to the nonlinear RNN dynamics are highly interpretable. In particular, they all involve approximate low-dimensional line attractor dynamics-a useful dynamical feature that can be implemented by linear dynamics and used to store an analog value BID10 . Furthermore, we show that this mechanism is surprisingly consistent across a range of RNN architectures. In this work we applied dynamical systems analysis to understand how RNNs solve sentiment analysis. We found a simple mechanismintegration along a line attractorpresent in multiple architectures trained to solve the task. Overall, this work provides preliminary, but optimistic, evidence that different, highly intricate network models can converge to similar solutions that may be reduced and understood by human practitioners.
We analyze recurrent networks trained on sentiment classification, and find that they all exhibit approximate line attractor dynamics when solving this task.
{"src": "scitldr-AIC_train", "paper_id": "SJlKDVS2hV"}