Bianchini and Scarselli [1]}, [2]} report upper and lower bounds of \(B(S)\)  for a series of network architectures (Table REF ).
In particular, Bianchini and Scarselli [1]}, [2]} demonstrate that \(B(S)\)  of a single-hidden-layer network grows polynomially with respect to the hidden layer width. That is, \(B(S) \in O(m^d)\) , where \(m\)  is the width of the hidden layer. They also prove that \(B(S)\)  of a deep neural network grows exponentially in the total number of hidden neurons. That is, \(B(S) \in O(2^M)\) , where \(M\)  is the total number of hidden neurons. This indicates that deep neural networks have higher expressive capacity, and therefore are able to learn more complex functions than shallow ones.
<TABLE>