Adaptation EvaluationWe also follow our Open Instruct evaluation suite Wang et al. (2023); Ivison et al. (2023) to evaluate OLMo after instruction fine-tuning and DPO training using our Wefocus on evaluations around model chat capabilities and safety to showcase the efficacy of using OLMo as a base for further fine-tuning.