The MSEs for the different models were: 0.425 (Linear regression), 0.342 (Random Forest) and 0.456 (Tree).The random forest was significantly better than the other models based on an adjusted t-test of the performance difference [1]},
with a 95% confidence interval of  for the difference to the linear model MSE and [-0.158;-0.071]
for the difference to the decision tree.
We reused the 15 random forests from the bootstrap to estimate the learner-PD and learner-PFI including their confidence intervals based on adjusted variance estimates.
Figure REF , top row, shows that the most important features were alcohol, sulphates and volatile acidity.
The model-PFI quantifies how important each feature was for a fixed random forest, and the confidence intervals show the variance of the approximation of the model-PFI due to Monte Carlo integration.
The model-PFI, however, cannot tell us how much the estimate varies due to model variance.
The learner-PFI quantifies this model variance.
Both model-PFI and learner-PFI gave a similar ordering for the top features.
The learner-PFI shows that alcohol is more important than sulphates (with no overlap in the confidence intervals), for which the model-PFI would suggest that the importance is almost equal.
