For the continuous control experiments, we present an additional comparison against the NGU [1]} and RIDE [2]} methods, which, as shown in Table REF , also employ episodic counts for their intrinsic reward signals. In continuous state spaces, computing exact state counts is poorly informative, as two states are rarely identical. Thus, we implement episodic counts as pseudo-counts following the procedure presented in NGU for both methods. We employ inverse-dynamics features for computing pseudo-counts for NGU while we directly use the states for RIDE.
