In contrast, Bayesian surprise [1]} can be defined as the information gained about a random variable, by observing another random variable. For instance, we can compute the information gained about the parameters of the model \(\theta \)  by observing new states as \(\mathcal {I}(\theta ; s_{t+1}|s_t, a_t)\) . Such signal has been used for exploration exploiting Bayesian neural networks [2]}, where Bayesian surprise is obtained by comparing the weights distribution before and after updating the model with newly collected states. However, this procedure is extremely expensive, as it requires an update of the model for every new transition. Alternatively, an approximation of Bayesian surprise is obtainable by using the variance of an ensemble of predictors [3]}, [4]}, though this method still requires to train several models.
