The effect of these four aspects on expressive capacity can be understood through the influence on the hypothesis space.
A selected model framework corresponds to a hypothesis space \(\mathcal {H}\) . Each hypothesis \(h\in \mathcal {H}\)  represents a model with the given framework. Once the model size is determined, the hypothesis space shrinks to a subset of \(\mathcal {H}\) .
For example, suppose we set up two models with depth \(l_1\)  and \(l_2\)  (\(l_1 \le l_2\) ), respectively, and both with width \(m\) , the corresponding hypothesis spaces are \(\mathcal {H}_1\)  and \(\mathcal {H}_2\) , respectively. We have \(\mathcal {H}_1 \subset \mathcal {H}_2\) . This is because, for each \(h \in \mathcal {H}_1\) , where \(h\)  is a hypothesis and thus a deep network model in this context, there exists \(h^{\prime }\in \mathcal {H}_2\)  whose first \(l_1\)  layers are identical to those in \(h\)  and the subsequent layers are identical mappings. It is easy to show that the expressive capacity of \(\mathcal {H}_1\)  will not exceed the expressive capacity of \(\mathcal {H}_2\) .
Recently, model framework and size and their effect on expressive capacity of deep models have been well explored [1]}, [2]}, [3]}, [4]}.
