Recall that \(W^\infty _p = W_p\)  and \(W^\infty = W\)  hold w.p.1 under Assumption REF . Thus, for any \((s,a) \in W_p\)  and \(s^{\prime } \in \lbrace  s^{\prime } \in W \;|\; T(s^{\prime } | s,a) > 0 \rbrace \) , we have \(T(\cdot \;|\; s,a) = \tilde{T}(\cdot \;|\; s,a)\)  and \(D(\cdot \;|\; s,a,s^{\prime }) = \tilde{D}(\cdot \;|\; s,a,s^{\prime })\)  in probability as \(|\mathcal {O}| \rightarrow \infty \)  by Theorem REF  under Assumptions REF  and REF . This implies \(Q^{\mathsf {Risk}} = \tilde{Q}^{\mathsf {Risk}}\) . Hence, for each \(s \in W\) , \(\tilde{\pi }_{win}(s)\)  maximizes \(Q^{\mathsf {Risk}}(s, \cdot )\)  on \(A_\varphi (s)\)  in probability. By Theorem REF  and the convergence property of Q-learning [1]}, under Assumption REF , there exists \(\gamma ^{\prime }\)  such that, for any \(\gamma > \gamma ^{\prime }\) , \(\tilde{\pi }^\gamma _{tr}\)  maximizes the satisfaction probability of \(\varphi _B\)  w.p.1. Therefore, there exists an optimal policy \(\pi ^\ast \in \Pi ^\ast _{\varphi _B \succ \mathsf {Risk}}\)  such that we have \(\tilde{\pi }^\ast _\gamma = \pi ^\ast \)  in probability.
