In addition to the RSI, we make some mild regularity assumptions on the function classes we study. We emphasize that none of these assumptions imply convexity or smoothness of the training objective \(\mathcal {L}_n ( + R(\)  or \(\mathrm {Err}_{n} (*)\) . First, we assume that the function class \(\mathcal {F}\)  is sufficiently smooth, having Lipschitz continuous gradients [1]}. Note that we study the case when the function class \(\mathcal {F}\)  is smooth, and do not impose this condition on the empirical risk, \(\mathcal {L}_n (\) , as is often assumed [2]}.
