While passively collecting and transcribing real-world dialogues from in-person clinical visits is feasible, two substantial challenges limit its effectiveness in training LLMs for medical conversations: (1) existing real-world data often fails to capture the vast range of medical conditions and scenarios, hindering its scalability and comprehensiveness; (2) the data derived from real-world dialogue transcripts tends to be noisy, containing ambiguous language (including slang, jargon, and sarcasm), interruptions, ungrammatical utterances, and implicit references. This in turn, may limit AMIE's knowledge, capabilities, and applicability.