<jupyter_start><jupyter_text>Microsoft Interpret ML によるモデル開発とモデル解釈製造プロセスから採取されたセンサーデータと検査結果のデータを用いて、品質管理モデルを構築します。  1. データ準備<jupyter_code>import pandas as pd
from sklearn.model_selection import train_test_split

df = pd.read_csv("data/Factory.csv")
from sklearn.model_selection import train_test_split

X = df.drop(columns=["Quality","ID"],axis=1)
y = df["Quality"].values

X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.1,random_state=100,stratify=y)<jupyter_output><empty_output><jupyter_text>2. データ探索<jupyter_code>from interpret import show
from interpret.data import ClassHistogram

hist = ClassHistogram().explain_data(X_train, y_train, name = 'Train Data')
show(hist)<jupyter_output><empty_output><jupyter_text>3. Explainable Boosting Machine (EBM) によるモデル学習<jupyter_code>from interpret.glassbox import ExplainableBoostingClassifier, LogisticRegression, ClassificationTree, DecisionListClassifier
seed = 1234
ebm = ExplainableBoostingClassifier(random_state=seed)
ebm.fit(X_train, y_train)   #Works on dataframes and numpy arrays<jupyter_output>WARNING - Passing a numpy array to schema autogen when it should be dataframe.<jupyter_text>4. モデル解釈 (グローバル)<jupyter_code>ebm_global = ebm.explain_global(name='EBM')
show(ebm_global)<jupyter_output><empty_output><jupyter_text>5. モデル解釈 (ローカル)テストデータから算出された予測値の解釈を行います。<jupyter_code>ebm_local = ebm.explain_local(X_test[:20], y_test[:20], name='EBM')
show(ebm_local)<jupyter_output><empty_output><jupyter_text>6. EBM モデルの精度確認<jupyter_code>from interpret.perf import ROC

ebm_perf = ROC(ebm.predict_proba).explain_perf(X_test, y_test, name='EBM')
show(ebm_perf)<jupyter_output><empty_output><jupyter_text>7. ロジスティック回帰、決定木モデルの学習<jupyter_code>from interpret.glassbox import LogisticRegression, ClassificationTree

# We have to transform categorical variables to use Logistic Regression and Decision Tree
X_enc = pd.get_dummies(X, prefix_sep='.')
feature_names = list(X_enc.columns)
X_train_enc, X_test_enc, y_train, y_test = train_test_split(X_enc, y, test_size=0.20, random_state=seed)

lr = LogisticRegression(random_state=seed, feature_names=feature_names, penalty='l1')
lr.fit(X_train_enc, y_train)

tree = ClassificationTree()
tree.fit(X_train_enc, y_train)<jupyter_output>/Users/konabuta/miniconda3/envs/explain/lib/python3.6/site-packages/sklearn/linear_model/logistic.py:433: FutureWarning:

Default solver will be changed to 'lbfgs' in 0.22. Specify a solver to silence this warning.<jupyter_text>8. ダッシュボードでのモデル比較<jupyter_code>lr_perf = ROC(lr.predict_proba).explain_perf(X_test_enc, y_test, name='Logistic Regression')
tree_perf = ROC(tree.predict_proba).explain_perf(X_test_enc, y_test, name='Classification Tree')

show(lr_perf)
show(tree_perf)
show(ebm_perf)<jupyter_output><empty_output><jupyter_text>9. モデル解釈の比較 (グローバル、ローカル)  Global Explain<jupyter_code>lr_global = lr.explain_global(name='LR')
tree_global = tree.explain_global(name='Tree')

show(lr_global)
show(tree_global)
show(ebm_global)<jupyter_output><empty_output><jupyter_text>Local Explainn<jupyter_code>lr_local = lr.explain_local(X_test[:20], y_test[:20], name='LR')
tree_local = tree.explain_local(X_test[:20], y_test[:20], name='Tree')

show(lr_local)
show(tree_local)
show(ebm_local)<jupyter_output><empty_output><jupyter_text>ダッシュボード<jupyter_code># Do everything in one shot with the InterpretML Dashboard by passing a list into show

show([hist, lr_global, lr_local, lr_perf, tree_global, tree_local, tree_perf, ebm_global, ebm_local, ebm_perf], share_tables=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1117696867061812, 'fraction_numerical': 0.012418854078464579, 'mean_word_length': 10.17981072555205, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '452621', 'license': 'MIT', 'n_tokens_mistral': 1533, 'n_tokens_neox': 1478, 'n_words': 216, 'path': 'Sample/Interpret/FactoryQC-InterpretML-classification.ipynb', 'repo_name': 'konabuta/DataExplore-Workshop', 'size': '3543'}