Image classification We visualize loss landscapes of MNIST [1]} to verify the LU mechanism, and study the dependence on training data size. Similar to the teacher-student case, we reduce losses and errors (one minus accuracy) to two variables (weight norm \(w\)  and data size \(N\) ) by minimizing over angular directions of weights, i.e.,
\(\leavevmode \xbox {resizebox}{\XMLaddatt {width}{384.2974pt}\tilde{l}_{\rm train}(w, N)\equiv l_{\rm train}(\mathbf {w}^*,N),\quad \tilde{l}_{\rm test}(w, N)\equiv l_{\rm test}(\mathbf {w}^*,N),\quad \mathbf {w}^*(w,N) \equiv \underset{||\mathbf {w}||_2=w}{\rm argmin}\ l_{\rm train}(\mathbf {w}, N),}\) 
