Data-Efficient Learning via Minimizing Hyperspherical Energy
Deep learning on large-scale data is dominant nowadays. The unprecedented
scale of data has been arguably one of the most important driving forces for
the success of deep learning. However, there still exist scenarios where
collecting data or labels could be extremely expensive, e.g., medical imaging
and robotics. To fill up this gap, this paper considers the problem of
data-efficient learning from scratch using a small amount of representative
data. First, we characterize this problem by active learning on homeomorphic
tubes of spherical manifolds. This naturally generates feasible hypothesis
class. With homologous topological properties, we identify an important
connection -- finding tube manifolds is equivalent to minimizing hyperspherical
energy (MHE) in physical geometry. Inspired by this connection, we propose a
MHE-based active learning (MHEAL) algorithm, and provide comprehensive
theoretical guarantees for MHEAL, covering convergence and generalization
analysis. Finally, we demonstrate the empirical performance of MHEAL in a wide
range of applications on data-efficient learning, including deep clustering,
distribution matching, version space sampling and deep active learning.