In this paper, we mainly focus on developing a similar depth-independent bound on sample complexity of a more general class of polynomial neural networks (PNNs), which is quite different from the architectures that previous works have considered. Specifically, we use \(\sigma (x)=x^k\ (k\ge 2)\)  as our activation function. There are two main reasons why PNNs are of particular interest in this paper. First, PNNs are easy to analyze because of the simple structure. [1]}, [2]} study the representation power of one instance of PNNs called \(\Pi \) -Nets. [3]}, [4]}, [5]} show that PNNs can efficiently approximate other networks. Specifically, it only requires \(\mathcal {O}(\log \log (1/\epsilon ))\)  polynomial activation layers to approximate a 2-layer sigmoid networks with \(l_1\)  regularization, where \(\epsilon \)  indicates the approximation error. Beside this, [3]} shows that the training technique for such networks is very similar to SGD, and can be computed efficiently for each iteration. The other reason is that we are still lacking an understanding of the generalization ability of PNNs. Therefore, our contribution in this paper is an in-depth study on the sample complexity of PNNs, and we show that the complexity can be depth-independent under certain conditions, which may shed some light in understanding the recent success of deep PNNs in image generation [7]} and face recognition [1]}. Note that most previous works on PNNs only focus on one specific choice of \(k\)  (i.e., \(k=2\) ), but our conclusion in this paper is applicable to any \(k\ge 2\) .
