As a comparison to our method, we choose the AdaGrad [1]} algorithm, as it is widely used for problems of type (REF ).
Both AdaGrad and CSG start each iteration with a presdescribed step size \(\eta _n>0\) , based on which the calculation of the true step size \(\tau _n\)  is performed (see alg:Backtracking). We want to test the stability of both methods with respect to the initially chosen step length.
For this purpose, we set \(\eta _n = \tfrac{\tau _0}{n^d}\) , where \(\tau _0>0\) , \(n\)  is the iteration count and \(d\in [0,1]\)  is fixed.
