When optimizing a model, knowing how much further performance could be improved given more training would be obviously helpful.
In statistical learning theory, this general quantity is referred to as Epistemic Uncertainty [1]}, of which there are multiple formal quantifications.
A naive approach is to use the variance of the model predictions as a proxy metric [2]}, [3]},
which does not distinguish between aleatoric and epistemic uncertainties. It is important to note that some authors like [2]} and [3]} refer to the combined quantity as epistemic uncertainty.
Conversely, evidential regression [6]} separates these types of uncertainties by proposing evidential priors over the data likelihood function.
However, this technique does not extend to nonstationary distributions, as is the case in policy conditioned \(Q\) -value predictions in RL.
More recently, [7]} also attempted to model aleatoric and epistemic uncertainties by relating them to the expected prediction errors of the model.
We postulate that this is a more epistemologically desirable way to measure epistemic uncertainty, as this relation directly predicts how much improvement can be gained given more data and learning capacity, whereas the evidential prior by [6]} is essentially a proxy metric.
