We denote the joint distribution induced by the data generating process as \({P}_{XY}\) , where \(X\)  is a \(p\) -dimensional random variable and \(Y\)  a 1-dimensional random variable.
We describe the true mapping from features \(X\)  to the target \(Y\)  with \(f(X) = {E}[Y|X=x]\) .
We denote a single random draw from the DGP with \(x^{(i)}\)  and \(y^{(i)}\) .
A dataset consisting of multiple draws from \({P}_{XY}\)  will be called \(n = \lbrace (x^{(1)},y^{(1)}), \ldots , (x^{(n)},y^{(n)})\rbrace \) , where \(n\)  is the number of samples and with each \((x^{(i)}, y^{(i)}) \sim {P}_{XY}\) , \(i \in \lbrace 1, \ldots ,n\rbrace \) .
An ML model \(\hat{f}\)  is a function (\(\hat{f}: \mathcal {X}\rightarrow \mathcal {Y}\) ) that maps a feature vector to a prediction (e.g. \(\mathcal {Y}= {R}\)  for regression).
The model \(\hat{f}\)  is induced based on a dataset \({n}\) , using a loss function \(L: \mathcal {Y}\times {R}^p \rightarrow {R}_0^+\) .
As the true function \(f\)  is unknown, the model \(\hat{f}\)  is interpreted instead of \(f\) , for example, with PD plots and PFI.
The model \(\hat{f}\)  is learned by an ML learner \(I: \Lambda \rightarrow \mathcal {H}\)  that maps from the space of datasets and the space of hyperparameters \(\Lambda \)  to the function hypothesis space \(\mathcal {H}\) .
The learning process contains two sources of randomness:
the training data being a random sample from \({P}_{XY}\)  and (possibly) the inherent randomness of the training process [1]}.For example, stochastic gradient descent and weight initialization in neural networks or bootstrap and feature sampling in random forests are sources of randomness.
Thus, a model \(\hat{f}\)  can be seen as realization of a random variable \(F\)  with distribution \({P}_{F}\) .
We assume that the model is evaluated with a risk function \(\mathcal {R}(\hat{f}) = {E}_{XY} [L(Y, \hat{f}(X))] = \int L(y, \hat{f}(x)) \text{d}{P}_{XY}\) , based on a loss function \(L\) .
To get unbiased estimates of the risk, model training and evaluation use different datasets.
The dataset \(n\)  is split into \({n_1}\)  for model training and \({n_2}\)  for evaluation, with \(n_1 + n_2 = n\) .
The empirical risk is estimated with \(\hat{\mathcal {R}}(\hat{f}_{{n_2}, \lambda }) := \frac{1}{n_2} \sum _{i=1}^{n_2} L\left(y^{(i)},\hat{f}_{{n_2}, \lambda }(x^{(i)})\right)\) .
