@inproceedings{
rakotomandimby2024learning,
title={Learning with Fitzpatrick Losses},
author={Seta Rakotomandimby and Jean-Philippe Chancelier and Michel De Lara and Mathieu Blondel},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=7Dep87TMJs}
}
Fenchel-Young losses are a family of convex loss functions,
encompassing the squared, logistic and sparsemax losses, among others.
Each Fenchel-Young loss is implicitly associated with a link function, for
mapping model outputs to predictions. For instance, the logistic loss is
associated with the soft argmax link function. Can we build new loss functions
associated with the same link function as Fenchel-Young losses?
In this paper, we introduce Fitzpatrick losses, a new family of convex loss
functions based on the Fitzpatrick function. A well-known theoretical tool in
maximal monotone operator theory, the Fitzpatrick function naturally leads to a
refined Fenchel-Young inequality, making Fitzpatrick losses tighter than
Fenchel-Young losses, while maintaining the same link
function for prediction.  
As an example, we introduce the Fitzpatrick logistic loss and the
Fitzpatrick sparsemax loss, counterparts of the logistic and the sparsemax
losses. This yields two
new tighter losses associated with the soft argmax and the sparse argmax,
two of the most ubiquitous output layers used in machine learning. We study in
details the properties of Fitzpatrick losses and in particular, we show that
they can be seen as Fenchel-Young losses using a modified, target-dependent
generating function. We demonstrate the effectiveness of Fitzpatrick losses for
label proportion estimation.
Learning with Fitzpatrick Losses
Seta Rakotomandimby Jean-Philippe Chancelier Michel De Lara Mathieu Blondel