Refinement describes the separability of a binary classification problem [1]}, [2]}. To build trust, it can be interpreted as the degree of confidence separation between correct and incorrect predictions. This is very similar to how calibration is assessed using predicted estimate as the assigned confidence. Good refinement indicates an ordinal ranking of predictions which allows better segregation of correct predictions from incorrect ones [3]}. Such a ranking can then allow the user to find an appropriate operating point. Moreover, it also plays an important part in describing predictors' effectiveness. From Figure REF  we can qualitatively infer that it is possible for a network to exhibit varying degree of calibration and refinement in its predictions for the same final accuracy (\(\approx 50\%\) ). In (a) we have a classifier which is well calibrated but poorly refined. As the network makes prediction mostly with a confidence of \(40\% - 60\%\)  with a matching accuracy, the usefulness of such a predictor is low. For (b), we see that the predictions are well separated but not well calibrated. We can select an operating threshold for the network to ensure that we don't encounter many false-positives in practice; however, the remaining predictions become uncalibrated. Case (c) shows an ideal scenario where the predictions are well separated and calibrated. The correct predictions are all predicted with very high confidence, and incorrect predictions consist of very low confidence values. We also present a real scenario via. (d, e) wherein the the confidence decreased after label smoothing has led to larger degradation of the quality of predictions.
