If a model is operating in the same way as the equivalent correct pushdown automaton, in theory, it should be able to recognize a string of any length [1]}, [2]}. This provides an excellent way of analyzing the limitations of RNNs on longer strings. We created a separate test set containing 1500 string samples of length (\(105 < T\)  and \(T > 161\) ). This sampling of lengths is important in order to properly test and understand the limitations of RNNs, since interpretability is largely dependent on the fact that the RNN acts as a pushdown automaton (PDA). This is crucial if our aim is to extract a minimal PDA from the final, trained model weights [3]}, [4]}, [5]}, [6]}, [7]}.
