Theorem 3.1 ([1]}) 
For any \(p\ge 1\)  and any net \(\mathsf {N}^{(d)}\)  such that \(\prod _{j=1}^d{W_j}\ge \Gamma \)  and \(\prod _{j=1}^d{W_j}_p \le M \) , for \(\forall r \in [d]\) , there exists an alternative net \(\widetilde{\mathsf {N}}^{(d)} \)  such that they are identical except for weights of the \(r\) -th layer: \(\widetilde{W}_r\) .
\(\widetilde{W}_r = s\mathbf {u}\mathbf {v}^T \) , where \(s, \mathbf {u}, \mathbf {v}\)  are the most significant singular value and corresponding left and right singular vectors of \(W_r\) .
\( \sup _{\mathbf {x}\in \mathcal {X}} *{\mathsf {N}^{(d)}(\mathbf {x}) - \widetilde{\mathsf {N}}^{(d)}(\mathbf {x}) } \le B \prod _{j=1}^d {W_j} *{\frac{2p\log (M/\Gamma )}{r}}^{1/p}. \) 

