@inproceedings{
kapturowski2023unlocking,
title={Unlocking the Power of Representations in Long-term Novelty-based Exploration},
author={Steven Kapturowski and Alaa Saade and Daniele Calandriello and Charles Blundell and Pablo Sprechmann and Leopoldo Sarra and Oliver Groth and Michal Valko and Bilal Piot},
booktitle={Second Agent Learning in Open-Endedness Workshop},
year={2023},
url={https://openreview.net/forum?id=HBvDqNF7sn}
}
We introduce Robust Exploration via Clustering-based Online Density Estimation (RECODE), a non-parametric method for novelty-based exploration that estimates visitation counts for clusters of states based on their similarity in a chosen embedding space. By adapting classical clustering to the nonstationary setting of Deep RL, RECODE can efficiently track state visitation counts over thousands of episodes. We further propose a novel generalization of the inverse dynamics loss, which leverages masked transformer architectures for multi-step prediction; which in conjunction with RECODE achieves a new state-of-the-art in a suite of challenging 3D-exploration tasks in DM-HARD-8. RECODE also sets new state-of-the-art in hard exploration Atari games, and is the first agent to reach the end screen in Pitfall!
Unlocking the Power of Representations in Long-term Novelty-based Exploration
Steven Kapturowski Alaa Saade Daniele Calandriello Charles Blundell Pablo Sprechmann Leopoldo Sarra Oliver Groth Michal Valko Bilal Piot