Attributing AUC-ROC to Analyze Binary Classifier Performance
Area Under the Receiver Operating Characteristic Curve (AUC-ROC) is a popular
evaluation metric for binary classifiers. In this paper, we discuss techniques
to segment the AUC-ROC along human-interpretable dimensions. AUC-ROC is not an
additive/linear function over the data samples, therefore such segmenting the
overall AUC-ROC is different from tabulating the AUC-ROC of data segments. To
segment the overall AUC-ROC, we must first solve an \emph{attribution} problem
to identify credit for individual examples.
  We observe that AUC-ROC, though non-linear over examples, is linear over
\emph{pairs} of examples. This observation leads to a simple, efficient
attribution technique for examples (example attributions), and for pairs of
examples (pair attributions). We automatically slice these attributions using
decision trees by making the tree predict the attributions; we use the notion
of honest estimates along with a t-test to mitigate false discovery.
  Our experiments with the method show that an inferior model can outperform a
superior model (trained to optimize a different training objective) on the
inferior model's own training objective, a manifestation of Goodhart's Law. In
contrast, AUC attributions enable a reasonable comparison. Example attributions
can be used to slice this comparison. Pair attributions are used to categorize
pairs of items -- one positively labeled and one negatively -- that the model
has trouble separating. These categories identify the decision boundary of the
classifier and the headroom to improve AUC.