We augment the metadata to 200 by mix-and-matching use cases and skills from different instructions. We randomly sample one use case from \(\{u_{i}\}_{i=1}^{n}\), and pair it with one or more skills sampled without replacement from \(\bigcup_{i=1}^{n}\mathbf{s}_{i}\). Although most skills are generalizable between use cases, we still conduct manual sanity check to exclude unreasonable use case and skills pairs. We align our hyper-parameters for iteratively improving instructions via Self-Rubrics with prior work (Xu et al., 2023): We generate 4 rubrics and corresponding actions, and at each iteration, we randomly choose 1 action for improving instruction. For fair comparison with WizardLM, we also use at most 4 improve iterations for each instruction (we count basic prompt generation as the first iteration). For Contrastive Filtering, we always use the strong LLM itself as the scorer. We set the scoring scale to 10 and the filtering threshold to 3 for all experiments. We obtain the threshold by developing on the AlpacaEval Dubois et al. (2023) dataset. And we find this threshold works generally well across different settings. Moreover, for LLaMA-based models, using their Alpaca Taori et al. (2023) counterparts as the target LLM for response generation in Contrastive Filtering works better than the original model that is not instruction tuned. For metadata extraction, base instruction generation and Self-Rubrics, we use a inference temperature of 0.7. We set the maximum number of tokens for generation to 2048 for LLaMA-based models, and 1024 for PaLM-based models due to API constraints. Moreover, although we set aside 20% validation set for metadata extraction, we still report the performance on the full test set in the main paper, the reasons are as follows: (1) We observe removing the validation set from the full test benchmark will not change the relative superior performance of our method, the performance gap between our method and baselines remains almost the same. Therefore, we keep them in for better reproducibility. (2) By carefully checking the generated instructions, we notice that none of the generated instructions overlap with the original validation instructions, so no data leaking happens during the data generation process.