We discuss the derivation of CPO objective, beginning with an analysis of Direct Preference Optimization (DPO) (Rafailov et al., 2023). DPO represents a more direct optimization objective utilized in reinforcement learning from human feedback (RLHF) (Ziegler et al., 2019; Ouyang et al., 2022). Given a set of source sentences \(x\), alongside preferred translation targets \(y_{w}\) and less preferred ones \(y_{l}\), we can access a static dataset of comparisons, denoted as \(\mathcal{D}=\left\{x^{(i)},y_{w}^{(i)},y_{l}^{(i)}\right\}_{i=1}^{N}\).