We conduct comprehensive experiments to evaluate CodecLM using different LLMs on multiple representative benchmarks, closely following well-established evaluation settings for open-domain instruction following in prior work (Xu et al., 2023; Chen et al., 2023). We also conduct a case study in Appendix A.8 to illustrate how CodecLM tailors an instruction step by step.