Montufar et al. [1]} also prove that, for the rank-\(k\)  Maxout activation function, the expressive capacity of a one-hidden-layer Maxout network with \(m\)  neurons is bounded by
\(MEC(\mathcal {N}) \ge k^{\min (d, m)}\) 
and \(MEC(\mathcal {N}) \le \min \lbrace \sum _{j=1}^{d} \tbinom{k^2m}{j},\ k^m\rbrace \) .
A rank-\(k\)  Maxout network, which consists of \(l\)  hidden layers and the width of each layer equals \(m\) , can compute any piecewise linear function with \(\Omega (k^{l-1}k^m)\)  linear regions. In conclusion, the maximum number of linear regions generated by a FCNN with piecewise linear activation functions increases exponentially with model depth.
