@inproceedings{
zhang2023largescale,
title={Large-scale Non-convex Stochastic Constrained Distributionally Robust Optimization},
author={Qi Zhang and Yi Zhou and Ashley Prater-Bennette and Lixin Shen and Shaofeng Zou},
booktitle={OPT 2023: Optimization for Machine Learning},
year={2023},
url={https://openreview.net/forum?id=ZLvIIkT1Fp}
}
Distributionally robust optimization (DRO) is a powerful framework for training robust models against data distribution shifts. This paper focuses on constrained DRO, which has an explicit characterization of the robustness level. Existing studies on constrained DRO mostly focus on convex loss function, and exclude the practical and challenging case with non-convex loss function, e.g., neural network. This paper develops a  stochastic algorithm and its performance analysis for non-convex constrained DRO. The computational complexity of our stochastic algorithm at each iteration is independent of the overall dataset size, and thus is suitable for large-scale applications. We focus on the general Cressie-Read family divergence defined uncertainty set which includes $\chi^2$-divergences as a special case. We prove that our algorithm finds an $\epsilon$-stationary point with an improved computational complexity  than existing methods. Our method also applies to the smoothed conditional value at risk (CVaR) DRO.
Large-scale Non-convex Stochastic Constrained Distributionally Robust Optimization
Qi Zhang Yi Zhou Ashley Prater-Bennette Lixin Shen Shaofeng Zou