Proof Sketch.: Consider any reward function \(r(x,y)\), which induces a corresponding optimal model \(\pi_{r}(y\mid x)\), specified by Eq. 4. We will show that a reward function from the equivalence class of \(r\) can be represented using the reparameterization given above.