is of Lebesgue-measure zero. Then every accumulation point of the sequence generated by CSG with constant stepsize \(\tau <\tfrac{2}{L}\)  is stationary and we have convergence in function values.

 Comparing theo:ConstantStepConvergence and cor:ConstantStepConvergence, observe that under the weaker assumptions on the set of stationary points of \(J\) , we no longer obtain convergence for the whole sequence of iterates. To illustrate why that is the case, consider the function \(J:\mathbb {R}^{d_{\textnormal {o}}}\rightarrow \mathbb {R}\)  given by \(J(u)=\cos (\pi \Vert u\Vert ^2)\)  and \(\mathcal {U}=\lbrace  u\in \mathbb {R}^{d_{\textnormal {o}}}\,:\,\Vert u\Vert ^2\le \frac{3}{2}\rbrace \) . Then, \(\mathcal {S}(J)=\lbrace 0\rbrace \cup \lbrace u\in \mathcal {U}\,:\,\Vert u\Vert =1\rbrace \) , i.e., every point on the unit sphere is stationary. Thus, we can not use lem:Convergence at the end of the proof to obtain convergence of \((u_n)_{n\in \mathbb {N}}\) . Theoretically, it might happen that the iterates \((u_n)_{n\in \mathbb {N}}\)  cycle around the unit sphere, producing infinitely many accumulation points, all of which have the same objective function value. This, however, did not occur when testing this example numerically.


While the assumption in cor:ConstantStepConvergence seems unhandy at first, there is actually a rich theory concerning such properties. For example, Sard's Theorem [1]} and generalizations [2]} give that the assumption holds if \(J\in C^{d_{\textnormal {o}}}\)  and \(\mathcal {U}\)  has smooth boundary. Even though it can be shown that there exist functions, which do not satisfy the assumption (e.g. [3]}, [4]}), such counter-examples need to be precisely constructed and will most likely not appear in any application.
