Montufar et al. [1]} focus on fully-connected feed forward neural networks (FCNNs) with piecewise linear activation functions (e.g., ReLU [2]} and Maxout [3]}), and propose to use the number of linear regions as a representation of model complexity (Figure REF ).
The basic idea is that an FCNN with piecewise linear activation functions divides the input space into a large number of linear regions. Each linear region corresponds to a linear function. The number of linear regions can reflect the flexibility and complexity of the model.
<FIGURE>