Neural Bregman Divergences for Distance Learning
Many metric learning tasks, such as triplet learning, nearest neighbor
retrieval, and visualization, are treated primarily as embedding tasks where
the ultimate metric is some variant of the Euclidean distance (e.g., cosine or
Mahalanobis), and the algorithm must learn to embed points into the pre-chosen
space. The study of non-Euclidean geometries or appropriateness is often not
explored, which we believe is due to a lack of tools for learning non-Euclidean
measures of distance. Under the belief that the use of asymmetric methods in
particular have lacked sufficient study, we propose a new approach to learning
arbitrary Bergman divergences in a differentiable manner via input convex
neural networks. Over a set of both new and previously studied tasks, including
asymmetric regression, ranking, and clustering, we demonstrate that our method
more faithfully learns divergences than prior Bregman learning approaches. In
doing so we obtain the first method for learning neural Bregman divergences and
with it inherit the many nice mathematical properties of Bregman divergences,
providing the foundation and tooling for better developing and studying
asymmetric distance learning.