It has been shown that the minimum size of shallow neural networks, such that \(\Phi \)  contains a given CPWL function \(f^*\) , directly depends on the number of affine pieces of \(f^*\)  whereas deep neural networks can “fold” the function and thus benefit of an exponential complexity reduction [1]}.
On the one hand, it is critical to determine the number of affine pieces in \(f^*\)  to figure out if shallow neural networks can solve the decoding problem.
On the other hand, when this is not the case, we can investigate if there exist preproccessing techniques to reduce the number of pieces in the CPWL function. We shall see that these preprocessing techniques are sequential and thus involve deep neural networks.
