We would also like to mention a different, but important, line of literature on solving stochastic objectives (with or without constraints) via sample-average approximation (SAA) methods. See, for example, [1]}, [2]}, [3]} for applying SAA on different problems and [4]}, [5]} for surveys. The SAA methods approximate a stochastic objective with some sampling schemes (such as Monte Carlo), and apply deterministic solvers for solving the approximated objective. In contrast, we consider stochastic approximation setup, where we apply a stochastic solver (i.e. StoSQP) for solving the original (stochastic) objective. The solver utilizes stochastic gradient and Hessian of the objective that are estimated by sampling.
