The above discussion suggests that in the presence of model miscalibration, using the tail of the predictions to inform the confidence score could be beneficial.As one illustrative example, consider a 10-class classification task and the predictions on two instances: \(f(x_1) = [0.9, 0.1, 0.0 , \dots 0.0]\)  and \(f(x_2) = [0.9, 0.1/9, \dots , 0.1/9]\) . We might expect that the confidence on \(x_1\)  should be higher than on \(x_2\) : intuitively, for \(x_1\)  the model is “deliberating” between two concrete options (the first and second classes) whereas for \(x_2\)  there is no clear alternative to the first class. However, by definition, \(\max _i f(x)_i\)  can make no such distinctions. Indeed, the literature on active learning (AL) has long since considered uncertainty scores that employ the rest of the probability vector [1]}. In the AL context, these are used to greedily select examples from a large pool of unlabeled examples for which labels will be requested. E.g., it is common to use the gap between the first and second largest entries of \(f(x)\) , and in some cases even the entropy of \(f(x)\) , to inform this selection.
