For LLaMA-based models, we follow the practices in instruction tuning in prior works Zhou et al. (2023); Chen et al. (2023). We use AdamW optimizer with \(\beta_{1}=0.9,\beta_{2}=0.95\) to finetune the target model for 15 epochs, as suggested by Zhou et al. (2023) for smaller data size. We set the initial learning rate to \(1\times 10^{-5}\) and linearly decaying to \(1\times 10^{-6}\) by the end of training. We set per GPU batch size to 8, which is equivalent to a total batch size of 64, as we use 8 A100 GPUs for training. The maximum token length is set to 2048.