While stopping criteria based on approximate first order conditions are rather straight-forward to implement thanks to lem:GradientError, we want to further discuss the step size dependency of SG methods. In most cases, finding an optimal step size requires knowledge of properties of the function which are usually not available. On the other hand, simpler step size rules, like constant step sizes, come with very strict assumptions with respect to (REF ). As an example, linear convergence rates for strongly convex objective functions with a finite sum structure (discrete measure \(\mu \)  in (REF )) were proven under the proposed strong growth condition in [1]}. However, this condition is (as stated by the authors) not satisfied in most applications, as it requires any optimal solution of (REF ) to be stationary for each individual component of the objective function. An overview of different convergence results for adaptive or fixed step size rules can be found in [2]}, [3]}, [4]}, [5]}, [6]}, [7]} and the references therein.
