Uncertainty Quantification in PDPs.
Quantifying the uncertainty of PDPs provides additional information about the reliability of the mean estimator.
[1]} quantified the model uncertainty specifically for random forests as surrogates in BO by calculating the standard deviation of the marginal predictions of the individual trees.
However, this procedure is not applicable to general probabilistic surrogate models, such as the commonly used GP.
There are approaches that quantify the uncertainty for ML models that do not provide uncertainty estimates out-of-the-box.
[2]} suggested a bootstrap approach for tree ensembles to quantify the uncertainties of effects based on PDPs.
Another approach to quantify the uncertainty of PDPs is to leverage the ICE curves. For example, [3]} implemented a method that marginalizes over the mean \(\pm \)  standard deviation of the ICE curves for each grid point.
However, this approach quantifies the underlying uncertainty of the data at hand rather than the model uncertainty, as explained in Appendix REF . A model-agnostic estimate based on uncertainty estimates for probabilistic models is missing so far.
