Concurrent to our work, Yuan et al. (2024) have developed an iterative framework for generating _new_ preference datasets and performing DPO training on the resulting datasets. They empirically demonstrated the superiority of their iterative framework in terms of AlpacaEval 2.0. In contrast, our work is complementary to the above in the sense that we focus on utilizing the _current_ preference data and does not undergo new data generation. Thus, our method can also be applied to Yuan et al. (2024) by changing the DPO training part to using sDPO instead. We leave the above combination as an interesting future work. Additionally, the evaluation used in Yuan et al. (2024) is also different to ours as we utilize tasks from Open LLM Leaderboard whereas Yuan et al. (2024) uses AlpacaEval 2.0.