Since the pioneering paper by Robbins and Monro [1]}, due to low demand for computer memory and cheap computation cost at every iteration, SA type algorithms becomes widely used in stochastic optimization and machine learning, see, e.g. [2]}, [3]}. If \(f(\cdot )\)  is twice continuously differentiable and strongly convex, in the classical analysis it is shown that the SA algorithm exhibits asymptotically optimal rate of convergence \(\mathbb {E}[f(x^k)-f^*]=O(k^{-1})\) , where \(x^k\)  is \(k\) th iterate and \(f^*\)  is the optimal value. An important improvement developed by Polyak [4]} and Polyak and Juditsky [5]} suggests that, longer stepsizes of SA algorithm can be adopted by consequently averaging the obtained iterates. Moreover, in [6]}
the authors show that, without assuming smoothness and strong convexity, a properly modified SA method achieves the convergence rate \(O(k^{-1/2})\)  and remarkably outperforms the sample average approximation (SAA) approach for a certain class of convex stochastic problems. After the seminal work [6]}, there are many significant results appeared, even for non-convex stochastic optimization problems, see [3]}, [9]} and references cited therein.
Among all mentioned works, the feasible set is an abstract closed convex set and none of these SA algorithms are applicable to expectation constrained problems. The main reason is that the computation of projection \(\Pi _{\Phi }\)  is quite easy only when \(\Phi \)  is of a simple structure.
However, when \(\Phi \)  is defined by (REF ), the computation is prohibitive.
