Multi layer perceptrons (MLPs) are known to be universal approximators that can approximate any continuous function on a compact set [1]}.
A MLP layer can be defined as a function \(f: \mathbb {R}^{d_{\mathrm {in}}} \rightarrow \mathbb {R}^{d_{\mathrm {out}}}\) :
\(f_{\mathbf {W}}(\mathbf {x}) = \mathrm {ReLU}(\mathbf {W}\mathbf {x})\) 
