Proof.: \(\pi_{w}\) represents an ideal policy that perfectly aligns the true data distribution of the preferred data. Hence, for any given data point \((x,y_{w},y_{l})\) from the preference dataset \(\mathcal{D}\), the conditions \(\pi_{w}(y_{w}|x)=1\) and \(0\leq\pi_{w}(y_{l}|x)\leq 1\) hold true.