We presentLLM-Blender, an ensembling framework designed to attain consistently superior performance by leveraging the diverse strengths of multiple open-source large language models (LLMs) . Our framework consists of two modules:PairRankerandGenFuser, addressing the observation that optimal LLMs for different examples can significantly vary.PairRankeremploys a specialized pairwise comparison method to distinguish subtle differences between candidate outputs. It jointly encodes the input text and a pair of candidates, using cross-attention encoders to determine the superior one. Our results demonstrate thatPairRankerexhibits the highest correlation with ChatGPT-based ranking.
Then,GenFuseraims to merge the top-ranked candidates, generating an improved output by capitalizing on their strengths and mitigating their weaknesses. To facilitate large-scale evaluation, we introduce a benchmark dataset,MixInstruct, which is a mixture of multiple instruction datasets featuring oracle pairwise comparisons.
OurLLM-Blendersignificantly outperform individual LLMs and baseline methods across various metrics, establishing a substantial performance gap.111https://yuchenlin.xyz/LLM-Blender222The experiments on summarization, translation, and constrained generation tasks in the prior version have been moved to the appendix. Instead, we mainly present our work in the context of instruction-following data and LLMs in this version.