Using these two complexity measures, Raghu et al. [1]} explore the performance of deep neural networks and report some findings. First, the effective complexity grows exponentially with respect to the depth of the model and polynomially with respect to the width. Fig. REF  is an example showing the evolution of a trajectory after each hidden layer of a deep network. Second, how the parameters are initialized affects the effective complexity. Third, injecting perturbations to a layer leads to exponentially larger perturbations at the remaining layers. Last, regularization approaches, such as Batch Normalization [2]}, help to reduce trajectory length. This explains why Batch Normalization helps model stability and generalization.
