Off-policy evaluation (OPE).    
OPE [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]} is a well-studied problem with several lines of research.
A natural way is trajectory-wise importance sampling (IS) [10]}.
This, however, suffers from its variance, which can grow exponentially with the horizon length [11]}, [12]}.
Marginal importance sampling methods [11]} are developed to address this problem by estimating the IS ratio between two stationary distributions, leading to estimators whose variance are polynomial w.r.t. horizon [14]}.
Though prior works can also be used to estimate the MIW,
they may not be directly applicable to our framework, since they either require knowing the behavior policy [11]}, [16]}, or
need to assume the MIW lying in the space of RKHS [17]}.
Another approach to estimate the MIW is through saddle-point optimization [1]}, [2]}, [20]}, [21]}, which can suffer from its complex optimization problem.
In this paper, we derive an fixed-point-style and behavior-agnostic MIW estimation method, which is simple to train and does not require additional assumptions.
