Model agnostic supervised local explanations (MAPLE) [1]} employs a tree ensemble, e.g., a random forest, to estimate the importance (the net impurity) of each feature. Feature selection is performed upfront on the background data by iteratively adding important features to a linear model until error is minimized on held out validation data. For local explanations, MAPLE learns a ridge regression model on the background data distribution with samples weighed by the tree leafs relevant to the explained instance.
