All three previous work use uni-directional language-model-like models. <cite>Linzen et al. (2016)</cite> start with existing sentences from wikipedia that contain a present-tense verb. <cite>They</cite> feed each sentence word by word into an LSTM, stop right before the focus verb, and ask the model to predict a binary plural/singular decision (supervised setup) or compare the probability assigned by a pre-trained language model (LM) to the plural vs singular forms of the verb (LM setup).