We implement the \(F\) -value estimation as in Eq. (REF ) on two off-policy algorithms, namely SAC and DQN. We then evaluate their behavior and performance in a variety of environments, from continuous control tasks to domain-randomized image-based observation environments available from Gym [1]} and PyBullet Gym [2]} libraries.
Following guidelines from RLiable [3]}, we report 50% interquartile means (IQM) with bootstrapped confidence intervals.
Our experiments aim to answer the following questions:
