Several successful approaches modeled intrinsic rewards as the `surprisal' of a model. In layman's terms, this can be described as the difference between the agent's belief about the environment state and the ground truth, and can be implemented as the model's prediction error [1]}, [2]}. However, searching for less predictable states suffers from the `NoisyTV problem', where watching a screen outputting white random noise appears more interesting than other exploratory behaviours [3]}. This because the noise of the TV is stochastic and thus results generally more interesting than the rest of the environment [4]}.
