the elicited history, diagnostic accuracy, the appropriateness of management plans and their rationale, and patient-centred considerations such as relationship-building, respect for the individual and communication efficacy [20]. If the conversational potential of LLMs is to be realized in medicine, there is a significant unmet need to better optimize development and evaluation of medical AI systems for characteristics such as these, which are unique to history-taking and diagnostic dialogue between clinicians and patients.