Mainly due to its superior performance, deep learning is disruptive in many applications, such as computer vision [1]}, natural language processing [2]} and computational finance [3]}. At the same time, however, a series of fundamental questions about deep learning models remain, such as why deep learning can achieve substantially better expressive power comparing to classical machine learning models,
how to understand and quantify the generalization capability of deep models, and how to understand and improve the optimization process. Model complexity of deep learning is a core problem and is related to many those fundamental questions.
