Like humans, large language models (LLMs)  do not always generate the best output on their first try. Motivated by how humans refine their written text, we introduce\ours, an approach for improving initial outputs fromLLMs through iterative feedback and refinement. The main idea is to generate an initial output using anLLM; then, the sameLLMprovidesfeedbackfor its output and uses it torefineitself, iteratively.\oursdoes not require any supervised training data, additional training, or reinforcement learning, and instead uses a singleLLMas the generator, refiner and the feedback provider.
We evaluate\oursacross 7 diverse tasks, ranging from dialog response generation to mathematical reasoning, using state-of-the-art (GPT-3.5and GPT-4) LLMs.
Across all evaluated tasks, outputs generated with\oursare preferred by humans and automatic metrics over those generated with the sameLLMusing conventional one-step generation, improving by20% absolute on average in task performance.
Our work demonstrates that even state-of-the-artLLMs like GPT-4 can be further improved at test-time using our simple, standalone approach.111Code and data athttps://selfrefine.info/.