Distributionally robustness optimization first appeared in the context of inventory management [1]} and has since been discovered by the machine learning community as a useful tool to train machine learning models which generalize better to new distributions [2]}, [3]}, [4]}.
The uncertainty set occurring in the distributionally robust loss has been studied in terms of Wasserstein balls in [3]}, [6]}, [4]}, [8]}, [9]}, [10]}, [11]}, [12]}, and \(f\) -divergence balls have been another popular choice to study distributional robustness [2]}, [14]}, [15]}, [16]}, [17]}.
From a more general viewpoint, [18]} connects integral probability metrics with distributional robustness and provides links with generative adversarial networks.
In another vein, maximum mean discrepancy measures have been investigated in [19]} for generalization in Kernel methods.
The authors of [6]} propose a method to certify generalization by using the dual formulation of the Wasserstein worst-case risk. However, their approach requires the loss model and loss function to be smooth and relies on an estimate of the Lipschitz constant of gradients, which quickly becomes loose for large problem sizes. Related techniques based on Wasserstein distances [3]}, [4]}, [12]}, [11]}, [8]} make similarly prohibitive assumptions and generally fail to provide scalable alternatives.
In contrast, we study uncertainty sets expressed as Hellinger balls and provide a model-specific distributional robustness guarantee which only makes minimal assumptions on the loss (boundedness) and thus scales to large problems. Orthogonal to this work is the topic of certified adversarial robustness [26]}, [27]}, [28]}, [29]}, [30]}. This line of research seeks to reason about robustness at the instance level, while we aim to bound the population loss over a set of distributions.
