Our key motivation was to corroborate and extend the results of <cite>Bryant and Briscoe (2018)</cite> to current state-of-the-art language models which have been trained in several languages and show that these models are tough baselines to beat for novel GEC systems.