The benefits of label smoothing have been recently highlighted by [1]}. We were able to shed light on a severe limitation of the approach, which practitioners were currently unaware of. Similar to LS, other easy to apply calibration methods are also damaging in practice. We observed that the degree of refinement degradation varies from one dataset to another. [2]} discussed the causes for miscalibration and accredited it to the over-fitting on the training data (under cross-entropy loss). We found that the training accuracy achieved by the baseline is \(99.99\%\) , \(99.4\%\)  and \(77.9\%\)  for CIFAR-10, CIFAR-100 and ImageNet respectively. This signals towards a comparably lower over-fitting in models trained on ImageNet and subsequently, a lower impact on calibration vs. refinement trade-off. A point to takeaway is that CIFARs are still relatively important in assessing impact on calibration and refinement for DNNs as they are helpful in evaluating family of over-fitted models.
