Another intuitive motivation for using \(Q_\pi ^{P^*}\)  as the test function is that the minimax loss Eq. (REF ) for optimizing \(\omega \)  is based on a saddle-point optimization, and \(f\in \mathcal {F}\)  is the dual variable for the MIW \(\omega \) .
From the previous discussion, MIW and the action-value function have some primal-dual relationship, and \(Q_\pi ^{P^*}\)  is the optimal “dual” variable for the off-policy evaluation problem w.r.t. the MIW \(\omega \) .
Since the dual variable \(f\)  in Eq. (REF ) is indeed some action-value function [1]}, we may set \(f\)  to be this optimal “dual” of \(\omega \) , which will lead to the choice of \(Q_\pi ^{P^*}\)  as the test function.
