A series of recent studies demonstrate that deep architectures significantly outperform shallow ones [1]}, [2]}. Depth efficiency [3]}, which is the effectiveness of depth of deep models, has attracted a lot of interest. Specifically, the studies on depth efficiency analyze why deep architectures can obtain good performance and measures the effects of model depth on expressive capacity. We divide the studies of depth efficiency into two subcategories: model reduction methods and expressive capacity measures.
