Strong benefits of finite width ( Blue Lines). Both with and without the intervention of \({\mathsf {+LR}}{\mathsf {+L2}}\) , the performance decreases monotonically and dramaticallyThis phenomenon was first observed in [1]}. towards or across the \({\mathsf {NTK}}\)  performance (Faded Orange Lines) when \(n\)  passes a threshold (\(n\approx 2^6/2^7\)  for NN/NN+).
