In this experiment our goal was to demonstrate that minimizing the AUM results in maximizing the AUC in the train set.
We used the chipseq data (a benchmark for labeled changepoint detection) from the UCI repository [1]}, treating each (set.name, fold) as a different train set, with pre-processing as previously described.github.com/tdhock/feature-learning-benchmark
In brief, for each labeled example, changepoint models were computed for a range of penalty values, which resulted in models with a range of changepoints (some with few changepoints, others with many).
Then the label error rate for each model was computed, along with a penalty \(\hat{\lambda }_i\)  which resulted in minimum label errors for each labeled example \(i\) .
Finally these penalty values were used as the initial prediction vector \(\mathbf {\hat{y}} = [ -\log \hat{\lambda }_1 \cdots -\log \hat{\lambda }_n]\) , which was used as the optimization variable in an AUM gradient descent algorithm.
A line search was used for the step size so that the AUM was guaranteed to decrease at each iteration.
Overall there were 68 different train sets with the number of labeled examples ranging from \(n=7\)  to 1011, and the total number of breakpoints in error functions ranging from \(B=47\)  to 3104.
