@inproceedings{
smit2023are,
title={Are we going {MAD}? Benchmarking Multi-Agent Debate between Language Models for Medical Q\&A},
author={Andries Smit and Paul Duckworth and Nathan Grinsztajn and Kale-ab Tessera and Thomas Barrett and Arnu Pretorius},
booktitle={Deep Generative Models for Health Workshop NeurIPS 2023},
year={2023},
url={https://openreview.net/forum?id=Bfr0m4Ucl6}
}
Recent advancements in large language models (LLMs) underscore their potential for responding to medical inquiries. However, ensuring that generative agents provide accurate and reliable answers remains an ongoing challenge. In this context, multi-agent debate (MAD) has emerged as a prominent strategy for enhancing the truthfulness of LLMs. In this work, we provide a comprehensive benchmark of MAD strategies for medical Q&A, along with open-source implementations. This sheds light on the effective utilization of various strategies including the trade-offs between cost, time, and accuracy. We build upon these insights to provide a novel debate-prompting strategy based on agent agreement that outperforms previously published strategies on medical Q&A tasks.
Are we going MAD? Benchmarking Multi-Agent Debate between Language Models for Medical Q   A
Andries Petrus Smit Paul Duckworth Nathan Grinsztajn Kale-ab Tessera Thomas D Barrett Arnu Pretorius