While there is a substantial amount of work on statistical (Rozovskaya and Roth, 2016; Junczys-Dowmunt and Grundkiewicz, 2014; Yannakoudakis et al., 2017) and neural (Ji et al., 2017; Xie et al., 2016; Yuan and Briscoe, 2016; Chollampatt et al., 2016; Chollampatt and Ng, 2017; Chollampatt and Ng, 2018) machine translation methods for GEC, we follow the approach of <cite>Bryant and Briscoe (2018)</cite> and explore how such models would fare in this task when treated as simple language models.