Instruction Following.Synthetic data can serve as a promising approach for training instruction-following models, particularly in scenarios where real-world data is scarce, expensive, or challenging to obtain. Self-instruct (Wang et al., 2022a) and Stanford Alpaca (Taori et al., 2023) are both using LLMs to generate instruction following data which covers a wide range of scenarios. They first pick a small set of "seed instruction following samples" and then ask the LLMs to imitate the format to generate more demonstrations. One concern of this type of method is how to keep the generated data high quality, which involves the complexity of queries (Liu et al., 2023d), the diversity of semantics (Ding et al., 2023), and the scale of the synthetic dataset (Yuan et al., 2023). To this end, Xu et al. (2023) propose Evol-Instruct which adds complexity to simple instructions via prompting. Mukherjee et al. (2023) leverage LLMs to revise the instructions and responses iteratively to include high-quality explanation traces in the FLAN dataset (Wei et al., 2022), and they find the trained model has improved performance in many NLP tasks. UltraChat (Ding et al., 2023) is large-scale and multi-round synthetic dialogue dataset, which is generated by two separate ChatGPT Turbo API models--one serves as the user role while the other serves as the assistant. They instruct the user model with carefully designed prompts to mimic real human user behaviors.