Since the release ofTülu, open resources for instruction tuning have developed quickly, from better base models to new finetuning techniques. We test and incorporate a number of these advances intoTülu, resulting inTülu2, a suite of improvedTülumodels for advancing the understanding and best practices of adapting pretrained language models to downstream tasks and user preferences. Concretely, we release: (1) Tülu-V2-mix, an improved collection of high-quality instruction datasets; (2) Tülu2,Llama-2 models finetuned on the V2 mixture; (3) Tülu2+DPO,Tülu2 models trained with direct preference optimization (DPO) , including the largest DPO-trained model to date (Tülu2+DPO 70B) ; (4) Code Tülu2,Code Llamamodels finetuned on our V2 mix that outperformCode Llamaand its instruction-tuned variant,Code Llama-Instruct. Our evaluation from multiple perspectives shows that theTülu2 suite achieves state-of-the-art performance among open models and matches or exceeds the performance of GPT-3.5-turbo-0301 on several benchmarks. We release all the checkpoints, data, training and evaluation code to facilitate future open efforts on adapting large language models.