A critical assumption for inference of effects and importance using interpretable machine learning is unbiasedness of the model.
The model bias is difficult to test, and can be introduced by, e.g. choice of model class, regularization and feature selection.
For example, regularization techniques such as LASSO introduce a small bias on purpose [1]} to decrease model variance and improve predictive performance.
We have to better understand how specific biases affect the prediction function and therefore PD and PFI estimates.
Another crucial limitation for inference of PD and PFI is the underestimation of variance due to data sharing between model refits.
While we could show that a simple correction of the variance [2]} vastly improves the coverage, a proper estimation of the variance remains an open issue.
A promising approach relying on repeated nested cross validation to correctly estimate the variance was recently proposed by [3]}.
However, this approach is more computationally intensive by up to a factor of 1,000.
