The memory- or speed- inefficiency can be resolved when \(\pi_{\text{ref}}\) is set as a uniform prior \(U\), as the terms \(\pi_{\text{ref}}(y_{w}|x)\) and \(\pi_{\text{ref}}(y_{l}|x)\) cancel each other out. This negates the need for additional computations and storage beyond the policy model itself.