CodecLM sets a new state-of-the-art on four open-domain instruction-following benchmarks with various LLM choices, demonstrating its effectiveness in LLM alignment for diverse instruction distributions.