The polynomial lower bound for width efficiency is smaller than the exponential lower bound for depth efficiency [1]}, [2]}, [3]}. That is, to approximate a deep model whose depth increases linearly, a shallow model requires at least an exponential increase in width. To approximate a shallow, wide model whose width increases linearly, a deep, narrow model requires at least a polynomial increase in depth. However, Lu et al. [4]} point out that depth cannot be strictly proved to be more effective than width since a polynomial upper bound for width is still lacking. The polynomial upper bound ensures that to approximate a shallow, wide model, a deep, narrow one requires at most a polynomial increase in depth.
