There are classical statistical learning methods which address the generalization error in machine learning models. Two notable approaches are VC dimension [1]}, [2]} and Rademacher complexity [3]}, [4]}.
In the former, the expected generalization loss is bounded by the square root of the VC dimension divided by the square root of \(n\) . In the latter, the expected generalization loss scales with the Rademacher complexity of the hypothesis class. In the case of a 2 layer neural network of width \(m\) , it is shown that both of these approaches result in a generalization bound of \(\mathcal {O}(\sqrt{m/n})\)  that is vacuous for overparameterized neural networks with a very large \(m\) .
