Active Learning Through a Covering Lens
Deep active learning aims to reduce the annotation cost for deep neural
networks, which are notoriously data-hungry. Until recently, deep active
learning methods struggled in the low-budget regime, where only a small amount
of samples are annotated. The situation has been alleviated by recent advances
in self-supervised representation learning methods, which impart the geometry
of the data representation with rich information about the points. Taking
advantage of this progress, we study the problem of subset selection for
annotation through a "covering" lens, proposing ProbCover -- a new active
learning algorithm for the low budget regime, which seeks to maximize
Probability Coverage. We describe a dual way to view our formulation, from
which one can derive strategies suitable for the high budget regime of active
learning, related to existing methods like Coreset. We conclude with extensive
experiments, evaluating ProbCover in the low budget regime. We show that our
principled active learning strategy improves the state-of-the-art in the
low-budget regime in several image recognition benchmarks. This method is
especially beneficial in semi-supervised settings, allowing state-of-the-art
semi-supervised methods to achieve high accuracy with only a few labels.