**LLM Backbones.** We adopt LLaMA-based Touvron et al. (2023) and PaLM-based Anil et al. (2023) LLMs as our target LLMs in our experiments. For LLaMA-based target LLMs, we use Gemini-Pro Team et al. (2023) as the strong LLM, and LLaMA-7B, -13B as the target LLMs. For PaLM-based target LLMs, we use text-unicorn as the strong LLM, and text-boson as the target LLM. PaLM-based models and Gemini-Pro are accessible through Google Cloud API1.