Lu et al. [1]} investigate the width efficiency of neural networks with ReLU activation function.
They extend the universal approximation theorem [2]}, [3]} to width-bounded deep ReLU neural networks. The classical universal approximation theorem [2]}, [3]} states that, one-hidden-layer neural networks with certain activation functions (e.g., ReLU) can approximate any continuous functions on a compact domain to any desired performance.
Lu et al. [1]} show that, for any Lebesgue-integrable function \(f:\mathbb {R}^d \rightarrow \mathbb {R}\)  and any \(\epsilon > 0\) , there exists a ReLU network \(\mathcal {N}: \mathbb {R}^d \rightarrow \mathbb {R}\)   that can approximate \(f\)  to \(\epsilon \)  \(L^1\) -distance. That is,
\(\nonumber \int _{\mathbb {R}^d} |f(x) - F_{\mathcal {N}}(x)|dx < \epsilon .\) 
