The standard approach in statistical learning theory [1]} is to select a hypothesis function \(f: \mathcal {X} \rightarrow \mathcal {Y}\)  from a pre-defined hypothesis class \(\mathcal {F}\)  to minimize the expected risk with respect to \(\mathcal {D}\)  and to solve the risk minimization problem:
\(\inf _{f \in \mathcal {F}} \mathcal {R}(f) := \mathbb {E}_{(x,y) \sim \mathcal {D}}[l(f(x),y)]\) ,
for a suitable choice of loss function \(l\) . In practice, we do not have access to the ground-truth distribution. Instead, we find an approximate solution by solving the empirical risk minimization (ERM) problem, in which case \(\mathcal {D}\)  is approximated by the empirical distribution \(\mathbb {P}_n = \frac{1}{n} \sum _{i=1}^n \delta _{z_i}\) . In other words, in ERM we solve the problem:
\(\inf _{f \in \mathcal {F}} \mathcal {R}_{n}(f) := \frac{1}{n} \sum _{i=1}^n l(f(x_i),y_i)\) .
