Arora et al. [1]} investigate the family of functions representable by deep neural networks with ReLU activation function.
They prove that every piecewise linear function \(f: \mathbb {R}^d \rightarrow \mathbb {R}\)  can be represented by a ReLU neural network that consists of at most \(\lceil \log _2(d+1)\rceil \)  hidden layers.
The family of piecewise linear functions is dense in the family of compactly supported continuous functions, and the family of compactly supported continuous functions is dense in Lebesgue space \(L^p(\mathbb {R}^d)\)  [1]}, [3]}. The Lebesgue space \(L^p(\mathbb {R}^d)\)  is defined as the the family of Lebesgue intergrable functions \(f\)  for which \(\int _{\mathbb {R}^d}|f| < +\infty \)  [3]}. Define \(L^p\)  norm as \(||f||_p = [\int _{\mathbb {R}^d}|f|^p]^{1/p}\)  [3]}. Then, the above conclusion can be extended to the \(L^p(\mathbb {R}^d)\) -space. That is, every function \(f \in L^p(\mathbb {R}^d)\)  can be approximated to arbitrary \(L^p\)  norm by a ReLU neural network which consists of at most \(\lceil \log _2(d+1)\rceil \)  hidden layers.
