To that end, we gave an overview of the existing systems and picked two for a concrete comparison on the same data, the CoMiC-EN system (Meurers et al., 2011a ) and the Texas system <cite>(Mohler et al., 2011)</cite> . In comparing the two, it was necessary to turn CoMiC-EN into a scoring system because the Texas corpus as the chosen gold standard contains numeric scores assigned by humans. Taking a step back from the concrete comparison, we gave a more general description of what is necessary to compare short answer evaluation systems.