Neural networks have shown an unprecedented success in the hands of practitioners in several fields. Examples of successful applications include classification [1]}, [2]} and generative modeling [3]}, [4]}. This practical efficiency has accelerated developing analytical tools for understanding the properties of neural networks. A long standing dilemma in the analysis of neural networks is their favorable generalization error despite typically being extremely overparameterized. It is well known that the classical statistical learning methods for bounding the generalization error, such as Vapnik–Chervonenkis (VC) dimension [5]}, [6]} and Rademacher complexity [7]}, [8]}, often result in vacuous bounds in the case of neural networks [9]}.
