Our choice of the test function is motivated by the following constrained optimization formulation of off-policy evaluation [1]}, [2]}:
\(\textstyle \begin{split}&\min _{Q}~~ (1-\gamma )\mathbb {E}_{s \sim \mu _{0}, a\sim \pi (\cdot \,|\,s)}\left[Q(s,a)\right] \\&\mathrm {s.t.}~~~~Q(s,a) \ge r(s,a) + \gamma \mathbb {E}_{s^\prime \sim P(\cdot \,|\,s, a), a^\prime \sim \pi (\cdot \,|\,s^\prime )}\left[Q(s^\prime , a^\prime )\right],\quad \forall \,(s,a) \in {\mathbb {S}}\times {\mathbb {A}}\,.\end{split}\) 
