**SFT and DPO** We also compare different training objectives. Given that CPO is designed to steer learning towards preferred data, a straightforward benchmark is to compare its performance against directly SFT on the same preferred data set. Furthermore, considering that CPO is an evolution of DPO, we also include a comparative analysis with DPO.