Furthermore, our experiments predominantly utilized SOLAR 10.7B models, driven by the state-of-the-art performance at the time of experimentation along with its unique 10.7 billion parameter size. The unique size of SOLAR 10.7B models made the usage of sDPO more necessary as there are far fewer open source LLMs that can be adopted as reference models.