Large language models (LLMs)  have emerged as valuable tools for many natural language understanding tasks. In safety-critical applications such as healthcare, the utility of these models is governed by their ability to generate outputs that are factually accurate and complete. In this work, we present dialog-enabled resolving agents (DERA) . DERA is a paradigm made possible by the increased conversational abilities of LLMs, namely GPT-4. It provides a simple, interpretable forum for models to communicate feedback and iteratively improve output. We frame our dialog as a discussion between two agent types – aResearcher, who processes information and identifies crucial problem components, and aDecider, who has the autonomy to integrate theResearcher’s information and makes judgments on the final output. We test DERA against three clinically-focused tasks. For medical conversation summarization and care plan generation, DERA shows significant improvement over the base GPT-4 performance in both human expert preference evaluations and quantitative metrics. In a new finding, we also show that GPT-4’s performance (70%)  on anopen-endedversion of the MedQA question-answering (QA)  dataset (, USMLE)  is well above the passing level (60%) , with DERA showing similar performance. We release the open-ended MEDQA dataset athttps://github.com/curai/curai-research/tree/main/DERA.