Moderate-sized large language models (LLMs)  – those with 7B or 13B parameters – exhibit promising machine translation (MT)  performance. However,
they do not match the performance of state-of-the-art conventional encoder-decoder translation models or larger-scale LLMs such as GPT-4. In this study, we bridge this performance gap. We first assess the shortcomings of supervised fine-tuning for LLMs in the MT task, emphasizing the quality issues present in the reference data, despite being human-generated. Then, in contrast to supervised fine-tuning which mimics reference translations, we introduceContrastive Preference Optimization (CPO) , a novel approach that trains models to avoid generating adequate but not perfect translations. Applying CPO to ALMAmodels with only 22K parallel sentences and 0.1% parameters yields significant improvements. The resulting model, calledALMA-R, can match or exceed the performance of the WMT competition winners and GPT-4 on WMT’21, WMT’22 and WMT’23 test datasets.111We release our code and models at:https://github.com/fe1ixxu/ALMA.