Recall (Eq. REF ) that for networks of width \(m\)  with linear last layer \(\Vert H\Vert =O(1/\sqrt{m})\) . On the other hand, it can be shown (e.g., [1]} and [2]} for shallow and deep networks respectively) that \(\lambda _{\text{m}in}(K)({\mathbf {w}}_0) = O(1)\)  and is essentially independent of the width.

Hence Eq. REF  guarantees that given any fixed radius \(R\) , for a sufficiently wide network \(\lambda _{\text{m}in}(K)({\mathbf {w}})\)  is separated from zero in the ball \({\mathcal {B}}_R\) . Thus the loss function satisfies the PL* condition in \({\mathcal {B}}_R\) . As we discussed above, this guarantees the existence of global minima of the loss function and convergence of gradient descent for wide neural networks with linear output layer.
