For PaLM-based models, we follow the default instruction tuning setting on Google Cloud's LLM tuning web UI. We set the number of tuning steps to 2000, the learning rate multiplier to 1, and use the TPU training option.