<jupyter_start><jupyter_text>Analyze Errors and Explore Interpretability of ModelsThis notebook demonstrates how to use the Responsible AI Widget's Error Analysis dashboard to understand a model trained on the regression boston housing dataset. The goal of this sample notebook is to predict housing prices with scikit-learn and explore model errors and explanations:1. Train an SVM classification model using Scikit-learn2. Run Interpret-Community's 'explain_model' globally and locally to generate model explanations.3. Visualize model errors and global and local explanations with the Error Analysis visualization dashboard.  Install Required Packages<jupyter_code># %pip install --upgrade interpret-community
# %pip install --upgrade raiwidgets<jupyter_output><empty_output><jupyter_text>Import required packages<jupyter_code>from sklearn.datasets import load_boston
from sklearn import svm

# Imports for SHAP MimicExplainer with LightGBM surrogate model
from interpret.ext.blackbox import MimicExplainer
from interpret.ext.glassbox import LGBMExplainableModel<jupyter_output><empty_output><jupyter_text>Load the wine data<jupyter_code>boston = load_boston()
X = boston['data']
y = boston['target']
feature_names = boston['feature_names']
# Split data into train and test
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=0)<jupyter_output><empty_output><jupyter_text>Train a SVM classification model<jupyter_code>clf = svm.SVR(gamma=0.001, C=100.)
model = clf.fit(X_train, y_train)
# Notice the model makes a fair amount of error
print("average abs error on test dataset: " + str(sum(abs(model.predict(X_test) - y_test))/y_test.shape[0]))<jupyter_output><empty_output><jupyter_text>Load simple ErrorAnalysis view without explanations<jupyter_code>from raiwidgets import ErrorAnalysisDashboard
predictions = model.predict(X_test)
ErrorAnalysisDashboard(dataset=X_test, true_y=y_test, features=feature_names, pred_y=predictions, model_task='regression')<jupyter_output><empty_output><jupyter_text>Train a surrogate model to explain the original blackbox model<jupyter_code>from interpret_community.common.constants import ModelTask
# Train the LightGBM surrogate model using MimicExplaner
model_task = ModelTask.Regression
explainer = MimicExplainer(model, X_train, LGBMExplainableModel,
                           augment_data=True, max_num_of_augmentations=10,
                           features=feature_names, model_task=model_task)<jupyter_output><empty_output><jupyter_text>Generate global explanationsExplain overall model predictions (global explanation)<jupyter_code># Passing in test dataset for evaluation examples - note it must be a representative sample of the original data
# X_train can be passed as well, but with more examples explanations will take longer although they may be more accurate
global_explanation = explainer.explain_global(X_test)
# Print out a dictionary that holds the sorted feature importance names and values
print('global importance rank: {}'.format(global_explanation.get_feature_importance_dict()))<jupyter_output><empty_output><jupyter_text>Visualize  Analyze model errors and explanations using Error Analysis dashboard<jupyter_code>from raiwidgets import ErrorAnalysisDashboard
ErrorAnalysisDashboard(global_explanation, model, dataset=X_test, true_y=y_test, model_task='regression')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0672463768115942, 'fraction_numerical': 0.00463768115942029, 'mean_word_length': 7.396593673965937, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '165261', 'license': 'MIT', 'n_tokens_mistral': 1032, 'n_tokens_neox': 980, 'n_words': 295, 'path': 'notebooks/individual-dashboards/erroranalysis-dashboard/erroranalysis-dashboard-regression-boston-housing.ipynb', 'repo_name': 'imatiach-msft/responsible-ai-widgets', 'size': '3450'}