While concurrent works [23] that focus on an iterative pipeline of generating _new_ preference data have been proposed, our method focuses on utilizing the _currently available_ preference datasets. Thus, our approach is complementary as sDPO can be easily applied to any preference data and further combination with concurrent works would be an exciting future direction.