<cite>Goldberg (2019)</cite> , following Linzen et al. (2016) , considered a correct prediction to be one in which the masked word receives a higher probability than other inflected forms of the lemma. This evaluation leaves open the possibility that selectional restrictions or frequency are responsible for the results rather than sensitivity to syntactic structure (Gulordava et al., 2018) . To remove this possibility, we take into account all words of the same part-of-speech as the masked word.