We found LBS to be working best in this benchmark, as it explores the most in-depth and the most efficiently in all tasks. The training curves are presented in Figure REF , averaging over runs with eight different random seeds. Further comparison against RIDE [1]} and NGU [2]}, which employ episodic counts to modulate exploration, are presented in Appendix.
