
Refinement enforces a separation between the two sets of prediction. [1]} provide an alternative definition of refinement for calibrated classifiers. We consider area under the ROC curve (\(r\) )  [2]}, as an appropriate choice of metric for measuring refinement of a model[3]}. A common interpretation of \(r\)  is that it denotes the expectation that a uniformly drawn random positive sample is ranked higher (higher confidence) than a uniformly drawn random negative sample. [4]} calculate \(r\)  as:
\(r = \frac{R^p-|S^n|\times (|S^n|+1)/2}{|S^p|\times |S^n|}\) 
