Language Models: We constructed two models, EduBERT and EduDistilBERT, which respectively refine <cite>BERT-base</cite> and DistilBERT , <cite>both of which</cite> were trained on general domain text from books and Wikipedia (<cite>Devlin et al., 2019</cite>) .