AdAUC: End-to-end Adversarial AUC Optimization Against Long-tail Problems
It is well-known that deep learning models are vulnerable to adversarial
examples. Existing studies of adversarial training have made great progress
against this challenge. As a typical trait, they often assume that the class
distribution is overall balanced. However, long-tail datasets are ubiquitous in
a wide spectrum of applications, where the amount of head class instances is
larger than the tail classes. Under such a scenario, AUC is a much more
reasonable metric than accuracy since it is insensitive toward class
distribution. Motivated by this, we present an early trial to explore
adversarial training methods to optimize AUC. The main challenge lies in that
the positive and negative examples are tightly coupled in the objective
function. As a direct result, one cannot generate adversarial examples without
a full scan of the dataset. To address this issue, based on a concavity
regularization scheme, we reformulate the AUC optimization problem as a saddle
point problem, where the objective becomes an instance-wise function. This
leads to an end-to-end training protocol. Furthermore, we provide a convergence
guarantee of the proposed algorithm. Our analysis differs from the existing
studies since the algorithm is asked to generate adversarial examples by
calculating the gradient of a min-max problem. Finally, the extensive
experimental results show the performance and robustness of our algorithm in
three long-tail datasets.