As shown on Figure REF , the MIWs estimated by the GenDICE variant degenerate, in a sense that most of the MIWs are \(\approx 0\)  at the begining and are 1 later on, with long tails on the MIW distributions.
This may explain the relatively bad performance of GenDICE.
Compared with GenDICE, DualDICE provides more diverse MIW estimates, which leads to its relatively better performance.
Unfortunately, Figure REF  shows that, for the DualDICE variant, the distribution of MIWs gradually concentrates on very small and very large values, which indicates the degeneration of the MIWs.
The VPM variant performs the best among these three alternatives, mainly because it provides relatively better MIW estimates in the second half of the training process.
However,
Figure REF  shows that the MIWs from the VPM variant are poorly-distributed at the beginning, though relatively better later on.
Our method provides well-behaved MIW estimates in the whole training process, which may explain our better result.
As shown by Figure REF , the MIWs from our method are well-shaped and concentrate around the mean 1 0 in the \(\log \left(\text{MIW}\right)\)  plots. over the entire training process.
In general, the well-shaping of the estimated MIW is important, since the Cramer-Rao lower bound of the mean-square-error for OPE is related to the square of the density ratio [1]}, [2]}.
