We also report evaluation results for ablated models in Tab. 2. 'SOLAR 10.7B + SFT + DPO' uses all the DPO data at once, _i.e.,_\(D_{1}+D_{2}\), same as the conventional DPO training setup. 'SOLAR 10.7B + SFT + sDPO Strat.' uses stratified sampling to sample \(\sim 16.67\%\) of the data points from the union of OpenOrca and Ultrafeedback Cleaned to form \(D_{1}\) and use the remaining \(\sim 83.33\%\) as \(D_{2}\) to mirror the dataset size of \(D_{1}\) and \(D_{2}\) used in SOLAR 10.7B + SFT + sDPO.