Calibration.
A binary classifier \(f: \rightarrow [0,1]\)  is said to be calibrated if \(\forall v \in [0,1]\) , \(\Pr [Y=1 \vert f(X) = v] = v\) . For classification problems with multiple classes (\(k > 2\) ), there are different ways to define calibration [1]}, [2]}.
Arguably the simplest and most popular approach is to restrict the attention only to the most likely prediction [3]}. According to this definition, a classifier \(f: \rightarrow \Delta ()\)  is calibrated if \(\forall v \in [0,1]\) , \(\Pr [Y = \arg \max _i f(X)_i \vert \max _i f(X)_i = v] = v\) .
