<jupyter_start><jupyter_text>Result Visualizations with Surrogate Explainers CHAPTER 02 - *Model Explainability Methods*From **Applied Machine Learning Explainability Techniques** by [**<NAME>**](https://www.linkedin.com/in/aditya-bhattacharya-b59155b6/), published by **Packt**  ObjectiveIn this notebook, we will try to implement some of the concepts related to Feature Importance and Sensitivity Analysis part of the Influence based explainability methods discussed in Chapter 2 - Model Explainability Methods.  Installing the modules Install the following libraries in Google Colab or your local environment, if not already installed.<jupyter_code>!pip install --upgrade pandas numpy matplotlib seaborn scikit-learn xgboost skope-rules<jupyter_output><empty_output><jupyter_text>Loading the modules<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt 
import seaborn as sns
sns.set(style="whitegrid")
import warnings
warnings.filterwarnings("ignore")
np.random.seed(5)
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier, plot_tree
from sklearn import tree
import xgboost as xgb
import sklearn
import six
import sys
sys.modules['sklearn.externals.six'] = six
from skrules import SkopeRules
from dtreeviz.trees import dtreeviz
import graphviz<jupyter_output><empty_output><jupyter_text>About the data **Adult Data Set - UCI Machine Learning Repository**This dataset is also known as the *Census Income* dataset which is used to predict whether the income exceeds $50k/year based on census data. It is a multivariate dataset used for classification based problems containing 14 different features. More details about this data can be found at - [https://archive.ics.uci.edu/ml/datasets/adult](https://archive.ics.uci.edu/ml/datasets/adult)  Loading the data<jupyter_code>data  = pd.read_csv('Datasets/adult_census.csv', header=None)
data.columns = ['Age','Workclass','fnlwgt','Education','Education-Num','Marital Status','Occupation','Relationship','Race','Sex','Capital Gain','Capital Loss','Hours per week','Country','AnnualIncome'].copy()
data.head()
data.shape
features = data.iloc[:,:-1]
labels = data.iloc[:,-1]
labels.value_counts()
labels = labels.apply(lambda x: 1 if x == ' >50K' else 0)<jupyter_output><empty_output><jupyter_text>Data Preprocessing<jupyter_code>features = pd.get_dummies(features)
features.head()
labels.value_counts()
x_train, x_test, y_train, y_test = train_test_split(features, labels, test_size=0.2, random_state=30)<jupyter_output><empty_output><jupyter_text>Training a model<jupyter_code>classifier = xgb.XGBClassifier(n_estimators=400, max_depth=10, base_score=0.5, objective='binary:logistic', random_state=30)
classifier.fit(x_train, y_train)
classifier.score(x_test, y_test)<jupyter_output><empty_output><jupyter_text>So, we have trained an XGBoost Classifier which is not inherently interpretable. Now, we will use certain Surrogate Explainers to explain black-box models. Let's start with RuleFit model. We will use the Skope-Rules framework: https://github.com/scikit-learn-contrib/skope-rules for this purpose.  RuleFit Surrogate Explainer<jupyter_code>skope_rules_clf = SkopeRules(feature_names=list(x_train.columns), random_state=100, 
                             n_estimators=4, max_depth=10, max_depth_duplication=8,
                             recall_min=0.03, precision_min=0.03, max_samples=0.85, bootstrap=True,
                             verbose=1, n_jobs=4)
skope_rules_clf.fit(x_train, y_train)
for rules in skope_rules_clf.rules_[:5]:
    print(rules)
    print("\n")<jupyter_output>('Age <= 61.5 and Age > 36.5 and Workclass_ Self-emp-not-inc <= 0.5 and Education-Num > 12.5 and Capital Gain <= 3120.0 and Marital Status_ Married-civ-spouse > 0.5 and Capital Loss <= 1794.0 and Occupation_ Other-service <= 0.5 and Hours per week > 32.5', (0.7257663628831814, 0.1626740947075209, 2))


('Age > 25.5 and Education-Num <= 14.5 and Education-Num > 12.5 and Capital Gain <= 3120.0 and Marital Status_ Married-civ-spouse > 0.5 and Capital Loss <= 1794.0 and Occupation_ Exec-managerial <= 0.5 and Occupation_ Farming-fishing <= 0.5 and Hours per week > 31.0', (0.6408268733850129, 0.13865076407007082, 2))


('Education-Num > 12.5 and Capital Gain > 5095.5 and Marital Status_ Married-civ-spouse > 0.5', (1.0, 0.08729861371300113, 2))


('Education-Num > 12.5 and Capital Gain > 5095.5 and Marital Status_ Married-civ-spouse > 0.5 and Occupation_ ? <= 0.5 and Occupation_ Farming-fishing <= 0.5', (0.9936170212765958, 0.08702944465150951, 2))


('Age <= 85.0 and Education-Num > 12.5 and[...]<jupyter_text>As we can observe from the above steps, for the Surrogate Explainer, we are fitting a separate model and not making use of the XGBoost model. Although the XGBoost model is more accurate, it is not interpretable. So, we try to use a less complex and interpretable model, like the RuleFit model, and can provide explainability to even complex models.  Decision Tree Surrogate Explainer<jupyter_code>clf = DecisionTreeClassifier(random_state=1234, max_depth=3)
clf.fit(features, labels)
# Visualizing Decision Trees
fig, axes = plt.subplots(figsize=(25,22))
tree.plot_tree(clf, filled = True)
plt.show()
viz = dtreeviz(clf, features, labels,
                target_name="AdultIncome",
                feature_names=features.columns,
                class_names=['>50K','<=50K'],
                scale=1.30)

viz<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1043605181536216, 'fraction_numerical': 0.051450465243568694, 'mean_word_length': 5.159550561797753, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 46, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '481655', 'license': 'MIT', 'n_tokens_mistral': 1986, 'n_tokens_neox': 1763, 'n_words': 503, 'path': 'Chapter02/Surrogate Explainer.ipynb', 'repo_name': 'PacktPublishing/Applied-Machine-Learning-Explainability-Techniques', 'size': '5494'}