In contrast, Bayesian surprise [1]} measures the difference between the posterior and prior beliefs of an agent, after observing new data. As we also show in this work, this means that for stochastic transitions of the environment, which carry no novel information to update the agent's beliefs, low intrinsic bonuses are provided, potentially overcoming the NoisyTV issue. Previous work adopting Bayesian surprise for exploration has mostly focused on evaluating surprise in the model's parameter space [2]}, which suffers from being computationally-expensive.
