<jupyter_start><jupyter_text>40-modeling-sklearn> Modeling using scikit-learnIn this notebook, we train models using the pipeline functionality from scikit-learn.<jupyter_code>#tables and visualizations
import pandas as pd
import numpy as np
import seaborn as sns

#machine learning
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer, make_column_selector
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, LabelBinarizer, StandardScaler
from sklearn import config_context
from sklearn.metrics import classification_report<jupyter_output><empty_output><jupyter_text>Load the data<jupyter_code># fill in the following filename with the path of your data.
data_filename = ''
#load the data and learn a bit
df = pd.read_csv(data_filename)
display(df.head())
df.info()<jupyter_output><empty_output><jupyter_text>Split the dataHere, we employ the initial split to separate the training from the golden holdout test set.  We may split this in a prior notebook to standardize across modeling strategies.  Make sure to uncomment `class_column` below and fill in the name of the column of interest.<jupyter_code>#class_column = 'target_column'
random_seed = 2435

X_train, X_test, y_train, y_test = train_test_split(df.drop(columns=class_column), df[class_column],
                                                   test_size=0.25, random_state=random_seed, stratify=df[class_column])<jupyter_output><empty_output><jupyter_text>Quick sanity check to make sure that everything seems OK:<jupyter_code># X Train
print('On X train: ')
print('X train dimensions: ', X_train.shape)
display(X_train.head())

# X test
print('\nOn X test: ')
print('X test dimensions: ', X_test.shape)
display(X_test.head())
# X Train
print('On y train: ')
print('y train dimensions: ', y_train.shape)
display(y_train.head())

# X test
print('\nOn y test: ')
print('y test dimensions: ', y_test.shape)
display(y_test.head())<jupyter_output><empty_output><jupyter_text>Create training pipelineThe example below uses logistic regression.<jupyter_code>#individual pipelines for differing datatypes
cat_pipeline = Pipeline(steps=[('cat_impute', SimpleImputer(missing_values=np.nan, strategy='most_frequent')),
                               ('onehot_cat', OneHotEncoder(drop='if_binary'))])
num_pipeline = Pipeline(steps=[('impute_num', SimpleImputer(missing_values=np.nan, strategy='mean')),
                               ('scale_num', StandardScaler())])     
#establish preprocessing pipeline by columns
preproc = ColumnTransformer([('cat_pipe', cat_pipeline, make_column_selector(dtype_include=object)),
                             ('num_pipe', num_pipeline, make_column_selector(dtype_include=np.number))],
                             remainder='passthrough')
#generate the whole modeling pipeline with preprocessing
pipe = Pipeline(steps=[('preproc', preproc),
                       ('mdl', LogisticRegression(penalty='elasticnet', solver='saga', tol=0.01))])

#visualization for steps
with config_context(display='diagram'):
    display(pipe)<jupyter_output><empty_output><jupyter_text>Cross-validation with hyperparameter tuning<jupyter_code>tuning_grid = {'mdl__l1_ratio' : np.linspace(0,1,5),
               'mdl__C': np.logspace(-1, 6, 3) }
grid_search = GridSearchCV(pipe, param_grid = tuning_grid, cv = 5, return_train_score=True)
grid_search.fit(X_train, y_train)
print(grid_search.best_score_)
grid_search.best_params_
pd.DataFrame(grid_search.cv_results_)<jupyter_output><empty_output><jupyter_text>Final fitThe final fit may be present due to the grid search.  If so, you can use the best estimator functionality below.<jupyter_code>grid_search.best_estimator_<jupyter_output><empty_output><jupyter_text>Variable importanceNow we assess the importance in the selected model to reveal any potential insights.<jupyter_code>grid_search.classes_
vip = grid_search.best_estimator_['mdl'].coef_[0]
vip
#get names in correct preproc order
cat_names = grid_search.best_estimator_.named_steps['preproc'].transformers_[0][1].named_steps['onehot_cat'].get_feature_names()
num_names = grid_search.best_estimator_.named_steps['preproc'].transformers_[1][2]

#create df with vip info
coef_info = pd.DataFrame({'feat_names':np.hstack([cat_names, num_names]), 'vip': vip})

#get sign and magnitude information
coef_info = coef_info.assign(coef_mag = abs(coef_info['vip']),
                             coef_sign = np.sign(coef_info['vip']))

#sort and plot
coef_info = coef_info.set_index('feat_names').sort_values(by='coef_mag', ascending=False)
sns.barplot(y=coef_info.index, x='coef_mag', hue='coef_sign', data=coef_info, orient='h', dodge=False);<jupyter_output><empty_output><jupyter_text>Performance metrics on test data<jupyter_code>print(classification_report(y_test, grid_search.best_estimator_.predict(X_test)))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10503194888178914, 'fraction_numerical': 0.0049920127795527154, 'mean_word_length': 5.918508287292818, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 38, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '146951', 'license': 'MIT', 'n_tokens_mistral': 1597, 'n_tokens_neox': 1576, 'n_words': 361, 'path': '40-modeling-sklearn.ipynb', 'repo_name': 'vanderbilt-data-science/python-hands-on', 'size': '5008'}