Supervised Fine-Tuning Supervised fine-tuning (SFT) requires sets of labeled input-output pairs. One of the most common SFT methods is instruction tuning (Wang et al., 2022; Mishra et al., 2021; Ouyang et al., 2022; Taori et al., 2023), which has emerged as one of the most powerful methods to improve model performance. With instruction tuning, the input is a natural language task description, and the output is an example of the desired behavior. Many current state-of-the-art LLMs have gone through instruction tuning after their pre-training phase.