I expected the attentionbased mechanism to fail on these (compared to the LSTM-based models), and am surprised by these results. The <cite>Gulordava et al. (2018)</cite> and Marvin and Linzen (2018) conditions rule out the possibility of overly relying on selectional preference cues or memorizing the wikipedia training data, and suggest real syntactic generalization is taking place.