@inproceedings{
yu2024rankrag,
title={Rank{RAG}: Unifying Context Ranking with Retrieval-Augmented Generation in {LLM}s},
author={Yue Yu and Wei Ping and Zihan Liu and Boxin Wang and Jiaxuan You and Chao Zhang and Mohammad Shoeybi and Bryan Catanzaro},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=S1fc92uemC}
}
Large language models (LLMs) typically utilize the top-k contexts from a retriever in retrieval-augmented generation (RAG).  In this work, we propose a novel method called RankRAG, which instruction-tunes a single LLM for both context ranking and answer generation in RAG.  In particular, the instruction-tuned LLMs work surprisingly well by adding a small fraction of ranking data into the training blend, and outperform existing expert ranking models, including the same LLM exclusively fine-tuned on a large amount of ranking data.  For generation, we compare our model with many strong baselines, including ChatQA-1.5, an open-sourced model with the state-of-the-art performance on RAG benchmarks.  Specifically,  our Llama3-RankRAG-8B and Llama3-RankRAG-70B significantly outperform Llama3-ChatQA-1.5-8B and Llama3-ChatQA-1.5-70B, respectively, on nine general knowledge-intensive benchmarks for RAG. In addition, it also performs comparably to GPT-4 on five RAG benchmarks in the biomedical domain without instruction fine-tuning on biomedical data, demonstrating its superb capability for generalization to new domains.
RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs
Yue Yu Wei Ping Zihan Liu Boxin Wang Jiaxuan You Chao Zhang Mohammad Shoeybi Bryan Catanzaro