Our method of improving sample efficiency with an oracle policy is loosely inspired by UCB [1]}.
Assume that we have a critic \(Q^\pi _\phi \)  and means of computing a measure of its likely prediction error \(e_\phi \) .
When presented with a state and action \(\lbrace \mathbf {s}_t, \mathbf {a}_t\rbrace \) , we can assign an upperbound to the true \(Q^\pi \)  value as in Eq. (REF ).
\( \Lambda (Q^\pi , \lbrace \mathbf {s}_t, \mathbf {a}_t\rbrace ) = Q^\pi _\phi (\mathbf {s}_t, \mathbf {a}_t) + e_\phi (\mathbf {s}_t, \mathbf {a}_t)\) 
