Our view is that, on a high level, there are two main approaches to achieve efficient exploration. The first one is to introduce the “curiosity" or exploration bonus as intrinsic reward [1]}, [2]}, [3]} for loss optimization. Bellemare et al. [4]} first proposed a pseudo-count-based exploration bonus as an intrinsic reward. Pathak et al. [5]} defined curiosity as a distance between the true next state feature vector and the predicted estimate of the next state feature vector. Thiede et al. [6]} brought curiosity-driven learning into the molecular generation task. In Thiede et al., curiosity is defined as a distance between the true reward of the current state and the predicted estimate of the current state reward.
However, these “curiosity-driven" intrinsic reward-based models sometimes fail to solve complex problems [7]}. The failures are explained as a result of the RL agent's detachment or derailment from the optimal solutions [8]}.
