It is well-known that generalization error is related to sample complexity of the function class of neural networks [1]}. Therefore, most works start from the analysis of sample complexity in neural networks. In [2]}, by inductively applying contraction principle on single layer of a neural network, it gives a bound on the network generalization error which has exponential dependence on the network depth \(d\) . With further investigation, [3]} shows that this bound can be tightened to polynomial dependence on \(d\) . However, in practice, people have observed that the number of training samples required for convergence does not necessarily goes up when a network becomes deeper. [4]} proves that, under certain conditions on norms of weight matrices and assumptions on the activation functions, we can achieve a depth-independent bound on the sample complexity.
