Unfortunately, we may not have good estimates for \(\Lambda (Q^\pi _\phi , (\mathbf {s}_t, \bar{\mathbf {a}}_t))\)  in out-of-distribution data.
One workaround that introduces additional machinery is to utilize Normalizing Flows [1]} and force high \(F\) -values for out-of-distribution data and regress to known values for in-distribution data.
Instead, we propose a benefit-of-doubt approach to measuring \(\Delta (Q^\pi , \lbrace \mathbf {s}_t, \mathbf {a}_t, \bar{\mathbf {a}}_t\rbrace )\) .
