We train the model in a _many-to-many_ multilingual machine translation manner, starting with ALMA-13B-LoRA as the initial checkpoint. During the training phase, we focus exclusively on updating the weights of the added LoRA parameters. These weights have a rank of 16 and only add an additional 12M parameters to the original 13B size of the model. We adhere to the default \(\beta\) value of 0.1 as suggested by Rafailov et al. (2023). The fine-tuning process of ALMA-13B-LoRA involves a batch size of 128, a warm-up ratio of 0.01, spanning a single epoch, and accommodating sequences with a maximum length of 512 tokens. To optimize training efficiency, we integrate the deepspeed tool (Rasley et al., 2020). We utilize the same prompt as Xu et al. (2023) and do not compute the loss for the prompt. While our primary focus is on the performance of 13B models, CPO markedly benefits 7B models as well. Consequently, we also release ALMA-7B-R and provide a detailed discussion of its performance in Appendix A.