**Auto-evaluation of Conversation Ratings.** We leveraged the model-based self-CoT auto-evaluation strategy to rate conversations on four evaluation axes from the PACES rubric, and validated that these auto-evaluation ratings were accurate and well aligned with the specialist ratings (Figures A.17 and A.18). Furthermore, to demonstrate that the inner self-play loop improved simulated dialogue quality, we applied the auto-evaluation method to the simulated dialogues generated before and after the self-play procedure. Results in Figure A.19 revealed that the simulated dialogues after self-play were preferred more often than the baseline dialogues without self-critique.