Automatic Short Math Answer Grading via In-context Meta-learning
Automatic short answer grading is an important research direction in the
exploration of how to use artificial intelligence (AI)-based tools to improve
education. Current state-of-the-art approaches use neural language models to
create vectorized representations of students responses, followed by
classifiers to predict the score. However, these approaches have several key
limitations, including i) they use pre-trained language models that are not
well-adapted to educational subject domains and/or student-generated text and
ii) they almost always train one model per question, ignoring the linkage
across a question and result in a significant model storage problem due to the
size of advanced language models. In this paper, we study the problem of
automatic short answer grading for students' responses to math questions and
propose a novel framework for this task. First, we use MathBERT, a variant of
the popular language model BERT adapted to mathematical content, as our base
model and fine-tune it for the downstream task of student response grading.
Second, we use an in-context learning approach that provides scoring examples
as input to the language model to provide additional context information and
promote generalization to previously unseen questions. We evaluate our
framework on a real-world dataset of student responses to open-ended math
questions and show that our framework (often significantly) outperforms
existing approaches, especially for new questions that are not seen during
training.