The piecewise linear property may provide novel opportunities for capturing model complexity in deep learning. In addition to the above studies on effective complexity, piecewise linear neural networks or the piecewise linear property can also help the exploration of expressive capacity (see Section 2, [1]}, [2]}, [3]}, [4]}).
Piecewise linear activation functions, especially ReLU, are popular and effective activation functions in many tasks and applications [1]}, [6]}. The local linear characteristic and a finite number of regional divisions facilitate quantifying and analyzing neural network model complexity with piecewise linear activation functions.
