**Introducing ALMA-7B-R** In this study, we extend the ALMA-13B-R training methodology to a 7B model size, specifically fine-tuning ALMA-7B-LoRA using the CPO method with the same preference data as ALMA-13B-R. Consistent with our findings from ALMA-13B-R, the application of CPO significantly enhances performance.