Large “instruction-tuned” language models (i.e., finetuned to respond to instructions)  have demonstrated a remarkable ability to generalize zero-shot to new tasks.
Nevertheless, they depend heavily on human-written instruction data that is often limited in quantity, diversity, and creativity, therefore hindering the generality of the tuned model.
We introduceSelf-Instruct, a framework for improving the instruction-following capabilities of pretrained language models by bootstrapping off their own generations.
Our pipeline generates instructions, input, and output samples from a language model, then filters invalid or similar ones before using them to finetune the original model.
Applying our method to the vanillaGPT3, we demonstrate a 33% absolute improvement over the original model onSuper-NaturalInstructions, on par with the performance of,111Unless otherwise specified, our comparisons are with thetext-davinci-001engine.
We focus on this engine since it is the closest to our experimental setup: supervised finetuning with human demonstrations. The newer engines are more powerful, though they use more data (e.g., code completion or latest user queries)  or algorithms (e.g., PPO)  that are difficult to compare with.which was trained with private user data and human annotations.
For further evaluation, we curate a set of expert-written instructions for novel tasks, and show through human evaluation that tuning GPT3 withSelf-Instructoutperforms using existing public instruction datasets by a large margin, leaving only a 5% absolute gap behind.Self-Instructprovides an almost annotation-free method for aligning pretrained language models with instructions, and we release our large synthetic dataset to facilitate future studies on instruction tuning.222Code and data are available athttps://github.com/yizhongw/self-instruct