Mhaskar et al. [1]} demonstrate that, to approximate a function \(f\in W^{NN}_{r,d}\)  to approximation degree \(\epsilon \) , a shallow network in \(S_m\)  requires \(O(\epsilon ^{-d/r})\)  trainable parameters. Meanwhile, to approximate a function \(f \in W^{NN}_{H, r, d}\)  to the same approximation degree, a network in \(D_m\)  requires only \(O(\epsilon ^{-2/r})\)  trainable parameters.
Then, Mhaskar et al. [1]} compare shallow Gaussian networks with hierarchical binary tree structures (Eq. (REF )) where each \(h_{ij}\)  computes a shallow Gaussian network, and obtain a similar conclusion. They demonstrate that functions with a designed compositional structure can be approximated by both deep and shallow networks to the same degree of approximation. However, the numbers of parameters of deep networks are much less than those of shallow networks.
