A ReLU neural network with \(n\)  inputs and \(W_1\)  neurons in the hidden layer can compute a CPWL function with at most \(\sum _{i=0}^{n}\binom{W_1}{i}\)  pieces [1]}.
This is easily understood by noticing that the non-differentiable part of \(\max (0,a)\)  is a \(n-2\) -dimensional hyperplane that separates two linear regions.
If one sums \(W_1\)  functions \(\max (0,d_{i} \cdot y)\) , where \(d_i\) , \(1\le i \le w_1\) , is a random vector, one gets \(W_1\)  of such \(n-2\) -hyperplanes.
The result is obtained by counting the number of linear regions that can be generated by these \(W_1\)  hyperplanes.
