Learning with Stochastic Orders
Learning high-dimensional distributions is often done with explicit
likelihood modeling or implicit modeling via minimizing integral probability
metrics (IPMs). In this paper, we expand this learning paradigm to stochastic
orders, namely, the convex or Choquet order between probability measures.
Towards this end, we introduce the Choquet-Toland distance between probability
measures, that can be used as a drop-in replacement for IPMs. We also introduce
the Variational Dominance Criterion (VDC) to learn probability measures with
dominance constraints, that encode the desired stochastic order between the
learned measure and a known baseline. We analyze both quantities and show that
they suffer from the curse of dimensionality and propose surrogates via input
convex maxout networks (ICMNs), that enjoy parametric rates. Finally, we
provide a min-max framework for learning with stochastic orders and validate it
experimentally on synthetic and high-dimensional image generation, with
promising results. The code is available at
https://github.com/yair-schiff/stochastic-orders-ICMN