This differs from Linzen et al. (2016) and <cite>Gulordava et al. (2018)</cite> by considering the entire sentence (excluding the verb) and not just its prefix leading to the verb, and differs from Marvin and Linzen (2018) by conditioning the focus verb on bidirectional context. I use the PyTorch implementation of BERT, with the pre-trained models supplied by Google.