However, DPO has notable drawbacks compared to common SFT. Firstly, DPO is **memory-inefficient**: it necessitates twice the memory capacity to simultaneously store both the parameterized policy and the reference policy. Secondly, it is **speed-inefficient**: executing the model sequentially for two policies doubles the processing time. To address these inefficiencies, we introduce contrastive preference optimization.