with an admissible set \(\mathcal {U}\) , a parameter set \(\mathcal {X}\) , \(j \in C^1(\mathcal {U}\times \mathcal {X};\mathbb {R})\)  and a (probability) measure \(\mu \) .
Such problems arise naturally in the context of machine learning and noisy simulations [1]}, [2]}. They also appear in other large scale optimization problems, such as topology or shape optimization, where one is interested in a design that is optimal for a continuous spectrum of outer parameters, e.g. [3]}, [4]}, [5]}. One usually does not consider deterministic approaches for the solution of this type of problems, as they are generally too computationally expensive or even intractable. Instead, one uses stochastic optimization schemes like stochastic gradient descent (SG) [6]} or stochastic average gradient descent (SAG) [7]}, to name the most prominent examples.
