Large Language Models For Instruction Fine-Tuning.The initial endeavors in instruction-following training work primarily focused on enhancing the language model's capacity for generalization across diverse tasks. This often involves the process of fine-tuning across substantially available Natural Language Processing datasets, and evaluates on the different NLP tasks. T5 [86] undertake the earliest attempts to train a range of NLP tasks, including Question and Answer, Document Summarization, and Sentiment Classification, by employing a consistent prompt format across all the data. Subsequently, instruction fine-tuning work such as FLAN [87], ExT5 [88], T0 [89], UnifiedQA [90], ZeroPrompt [91], and FLAN-T5 [92] emerged to adapt for a large number of downstream tasks. To address the challenge of misalignment between model outputs and human requirements, OpenAI manually annotates the instruction library to construct a diverse range of tasks. Simultaneously, Reinforcement Learning from Human Feedback technology is employed, which facilitate the rapid development of LLMs such as InstructGPT [2], ChatGPT\({}^{\ast}\), GPT-4 [3]. To reduce manual involvement, self-instruct [93] improves instruction-following through self-generated instructions. Alpaca [22] used a dataset of 50k instructions generated from a limited (e.g., 175 samples) seed set of manually-written instructions. Vicuna [23] used 70k user-shared conversations with ChatGPT collected from ShareGPT.com. Meanwhile, WizardLM [24] introduces the evol-instruct approach, which seeks to refine the existing instruction data by enhancing both its complexity and diversity.