In the Q-learning setting of [1]}, the DP constraint is applied on the Q-learning algorithm itself, viewed as a function \(\mathcal {A}: \mathcal {R} \rightarrow \mathcal {Q}\)  from the space of reward functions to the space of Q-functions. The requirement is then that for any reward functions \(r, r^{\prime }\)  with \(\Vert r - r^{\prime }\Vert _\infty < 1\) , \(\log p(\mathcal {A}(r)) - \log p(\mathcal {A}(r^{\prime })) < \epsilon \) , with a probability measure over \(\mathcal {Q}\)  provided by an RHKS. We qualitatively compare policies learned under this constraint to policies satisfying our MI constraint in section .
