ZeroR (majority class) 66.70 mBERT <cite>(Devlin et al., 2019)</cite> 90.21 BERTje (de Vries et al., 2019) 94.94 RobBERT (ours) 98.03 RobBERT outperforms previous models as well as other BERT models both with as well as without fine-tuning (see Table 1 and Table 2 ).