Model complexity of deep learning is concerned about, for a certain deep learning architecture, how complicated problems the deep learning model can express [1]}, [2]}, [3]}, [4]}. Understanding the complexity of a deep model is a key to precisely understanding the capability and limitation of the model. Exploring model complexity is necessary not only for understanding a deep model itself, but also for investigating many other related fundamental questions.
For example, from the statistical learning theory point of view, the expressive power of a model is used to bound the generalization error [5]}. Some recent studies propose norm-based model complexity [6]} and sensitivity-based model complexity [7]}, [8]} to explore the generalization capability of deep models. Moreover, detecting changes of model complexity in a training process can provide insights into understanding and improving the performance of model optimization and regularization [3]}, [4]}, [11]}.
