
\(M_{\delta }\) :
This is a high probability upper bound for the \(\ell _2\)  norm of the initialized vector \(\left\Vert w^{(0)}\right\Vert \) .
In practice, for commonly used initialization schemes,
such as Xavier initialization [1]} and Kaiming initialization [2]}, the norm \(\left\Vert w^{(0)}\right\Vert \)  is uniformly bounded with high probability.

\(R_{{n, \delta }}\) :
This is an high probability upper bound for the optimization path length over the initialization and training samples.
In theorem REF , we assume that such a bound exists, and show that it immediately yields
a generalization estimate based on this bound.
In theorem REF , we give a sufficient condition to obtain such path estimate.
The asymptotic analysis of \(R_{{n, \delta }}\)  is problem-dependent,
and we provide several examples in Section .

\(L_\ell (\mathcal {S}_{a, b})\)  and \(M_{a, b}\) : These quantities are directly related to the loss function \(\ell \) . For instance, given a loss function \(\ell : \mathbb {R} \times \mathbb {R} \xrightarrow{} [0, 1]\)  that is 1-Lipschitz in the first argument, we have that \(L_\ell (\mathcal {S}_{a, b}) = M_{a, b} = 1\) .

\(\left\Vert L_\Psi (\mathcal {S}_{a, b})\right\Vert \) : This term is related to the properties of \(f\) .
For example, when \(f\)  is linear, \(\Psi (x, y) = x + y\) , then \(\left\Vert L_\Psi (\mathcal {S}_{a, b})\right\Vert  = \sqrt{2}\) .

