Large language models (LLMs)  strengthen instruction-following capability through instruction-finetuning (IFT)  on supervised instruction/response data. However, widely used IFT datasets (e.g.,Alpaca’s 52k data)  surprisingly contain many low-quality instances with incorrect or irrelevant responses, which are misleading and detrimental to IFT. In this paper, we propose a simple and effective data selection strategy that automatically identifies and filters out low-quality data using a strong LLM (e.g., ChatGPT) . To this end, we introduceAlpaGasus,
which is finetuned on only 9k high-quality data filtered from the 52kAlpacadata.AlpaGasussignificantly outperforms the originalAlpacaas evaluated by GPT-4 on multiple test sets and the controlled human evaluation. Its 13B variant matchesperformance of its teacher LLM (i.e., Text-Davinci-003 generating the 52k data)  on test tasks. It also provides 5.7x faster training, reducing the training time for a 7B variant from 80 minutes (forAlpaca)  to 14 minutes111We apply IFT for the same number of epochs asAlpaca(7B)  but on fewer data, using 4NVIDIA A100 (80GB)  GPUs and following the originalAlpacasetting and hyperparameters.. Moreover, the experiments prove the efficacy of our method across diverse datasets, base models, and LLM filters. Overall,AlpaGasusdemonstrates a novel data-centric IFT paradigm that can be generally applied to instruction-tuning data, leading to faster training and better instruction-following models. Our project page is available at:https://lichang-chen.github.io/AlpaGasus/.