As a first attempt for solving expectation constrained stochastic optimization problems with stochastic approximation technique, Lan and Zhou [1]} introduce a cooperative stochastic approximation (CSA) algorithm for solving (REF ) with single expectation constraint (\(p=1\) ), which is a stochastic counterpart of Polyak's subgradient method [2]}. The authors show that CSA exhibits the optimal \(\mathcal {O}(1/\sqrt{K})\)  rate of expected convergence, where \(K\)  is a fixed iteration number. In an online fashion, Yu et al. [3]} proposed an algorithm (simply denoted by “YNW”) that can be easily extended to solve (REF ) with multiple expectation constraints. Under the Slater's condition and the assumption that \(\mathcal {C}\)  is compact, they show that the algorithm
can achieve \(O(1/\sqrt{K})\)  expected regret and \(O(\log (K)/\sqrt{K})\) 
high probability regret. In [4]}, the author proposes a penalized stochastic gradient (PSG) method and establishes its almost sure convergence and expected convergence rates. PSG can be roughly viewed as a hybrid of the classical penalty method for nonlinear programming and the stochastic quasi-gradient method [5]} for stochastic composition problem. In [6]}, a stochastic level-set method, which ensures a feasible solution path with high probability, is proposed and analyzed.
