In order to accurately emulate the ratings of specialists on the clinical criteria in our OSCE evaluation framework, we developed a model-based auto-evaluation procedure leveraging the AMIE model to score dialogues from 1 to 5 based on how well they exemplified those qualitative criteria. We initially focused on a subset of four clinical axes from the PACES criteria (see Table A.2), however, this procedure can be easily extended to other ratings.