In this study, we initially proposed the potential quality issues of gold references in the MT task, highlighting instances where advanced translation models can outperform these references. This finding not only challenges model training via SFT, but also the evaluation procedure that uses reference-based metrics. Subsequently, we introduce Contrastive Preference Optimization, a more efficient variant of of DPO. This method leverages both model-generated and reference data to guide the model in avoiding near-perfect yet flawed translations and learning superior ones. Our developed model, ALMA-13B-R, stands out as the first moderate-size LLM-based translation model to match, and in some cases surpass, the performance of GPT-4 and WMT competition winners, marking a significant advancement in the field of MT.