Post-hoc calibration.
Since many ML models are not typically calibrated “out of the box”, it is a common practice to post-process the model outputs in a way that improves their calibration. Methods include Histogram Binning [1]}, Isotonic Regression [2]} and Platt scaling [3]}. In the context of modern ML models, the method of Temperature Scaling (TS) has demonstrated to be both simple and effective [4]}. For classifiers of the form \(f(x) = \sigma ()\)  (where \(\)  is the logit vector and \(\sigma \)  is the softmax operator), TS rescales the logit vector \(\)  by a factor of \(T\)  before applying softmax \(\sigma \) . The hyperparameter \(T\)  is called the temperature as it has the effect of “softening” the softmax (i.e. increasing the entropy of the output probability vector) when \(T>1\) . In particular, \(T=1\)  recovers the original model output and as \(T \rightarrow \infty \)  the model output approaches \(1/K\) . \(T\)  is optimized by minimizing the Negative Log Liklihood on a labeled validation set.
