<jupyter_start><jupyter_text>```NoteBook Focus```---1. Train numerous models to select best models for hypertuning.2. Build a model on unbalanced data and compare it to a model with balanced data. I want to make sure the models are performing they way they should.    a. Unbalanced classes model should misclassify the minority class frequently.        b. Balanced classes model should have great score across the board.  ```Imports```---<jupyter_code>import pandas as pd
from classifiers_copy import classify
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

import warnings
warnings.filterwarnings('ignore')<jupyter_output><empty_output><jupyter_text>```Creating new dataframe with balanced classes```---<jupyter_code>drugs = pd.read_csv('../Claire/data/drugs_2020_simply_imputed.csv')
drugs.head()
drugs['prisdum'].value_counts(normalize=True)

# 0 = no prison time
# 1 = prison time
# checking for missing values Claire might've missed before modeling
drugs.isnull().sum()
# concatting new df with equal classes for modeling

# separate all minor classes
df_0 = drugs[drugs['prisdum']==0]
print(df_0.shape)

# separate all majority class and sample 754 to match minor class
df_1 = drugs[drugs['prisdum']==1]
df_1_sample = df_1.sample(n=754, replace=False)
print(df_1_sample.shape)

# concat both df's
equal_class_df = pd.concat([df_0,df_1_sample], ignore_index=True)
print(equal_class_df.shape)<jupyter_output>(754, 64)
(754, 64)
(1508, 64)<jupyter_text>```Modeling```---<jupyter_code># model with balanced classes

#set up X/y
X = equal_class_df.drop(columns='prisdum')
y = equal_class_df['prisdum']

# set train/test split
X_train, X_test, y_train, y_test = train_test_split(X,y,random_state=42, train_size=0.7, stratify=y)

# scale data
ss = StandardScaler()
X_train_ss = ss.fit_transform(X_train)
X_test_ss = ss.fit_transform(X_test)

# train multiple models
equal_class_scores = classify(X_train_ss,X_test_ss,y_train,y_test)
equal_class_scores
# import model
from sklearn.linear_model import LogisticRegression

# instantiate, train, evaluate
logreg = LogisticRegression()
logreg.fit(X_train_ss,y_train)
print(f"train acc: {logreg.score(X_train_ss,y_train)}")
print(f"test acc: {logreg.score(X_test_ss,y_test)}")

# create df to visualize coefs
logreg_scores = pd.DataFrame(columns=X.columns,data=logreg.coef_).T
logreg_scores[logreg_scores[0]>0].sort_values(by=0,ascending=False).head(10)
from sklearn.metrics import confusion_matrix, plot_confusion_matrix

# set up confusion matrix
cm = confusion_matrix(y_test,logreg.predict(X_test_ss))

# plot matrix
plot_confusion_matrix(logreg, X_test_ss, y_test, cmap='BuGn', display_labels=['no prison time', 'prison time']);<jupyter_output><empty_output><jupyter_text>**These results are great for a model that has balanced classess. We would never want to tell the person/offender the incorrect probability of getting a prison sentence.**<jupyter_code># model with UN_balanced classes

#set up X/y
X = drugs.drop(columns='prisdum')
y = drugs['prisdum']

# set train/test split
X_train, X_test, y_train, y_test = train_test_split(X,y,random_state=42, train_size=0.7, stratify=y)

# scale data
ss = StandardScaler()
X_train_ss = ss.fit_transform(X_train)
X_test_ss = ss.fit_transform(X_test)

# train multiple models
unbalanced_class_scores = classify(X_train_ss,X_test_ss,y_train,y_test)
unbalanced_class_scores
# import model
from sklearn.linear_model import LogisticRegression

# instantiate, train, evaluate
logreg = LogisticRegression()
logreg.fit(X_train_ss,y_train)
print(f"train acc: {logreg.score(X_train_ss,y_train)}")
print(f"test acc: {logreg.score(X_test_ss,y_test)}")

# create df to visualize coefs
logreg_scores = pd.DataFrame(columns=X.columns,data=logreg.coef_).T
# set up confusion matrix
cm = confusion_matrix(y_test,logreg.predict(X_test_ss))

# plot matrix
plot_confusion_matrix(logreg, X_test_ss, y_test, cmap='BuGn', display_labels=['no prison time', 'prison time']);<jupyter_output><empty_output><jupyter_text>**One of the other features is giving it away because the model should be misclassifying the minor class since the classes are very unbalanced. More EDA will be done to identify these features. The cm_plot basically states the model is predicting the minority class very well.**  ```More Modeling```---1. After some eda was done on the dataset we discovered that the feature 'suprdum' is 0.74 correlated to the target variable 'prisdum'. Modeling excluding this feature will be done to see how the model performs. We're expecting the models to be worst without this feature.<jupyter_code># model with UN_balanced classes and removal of highly correlated features

#set up X/y
X = drugs.drop(columns=['prisdum','suprdum'])
y = drugs['prisdum']

# set train/test split
X_train, X_test, y_train, y_test = train_test_split(X,y,random_state=42, train_size=0.7, stratify=y)

# scale data
ss = StandardScaler()
X_train_ss = ss.fit_transform(X_train)
X_test_ss = ss.fit_transform(X_test)

# train multiple models
unbalanced_class_scores_1 = classify(X_train_ss,X_test_ss,y_train,y_test)
unbalanced_class_scores_1
# import model
from sklearn.linear_model import LogisticRegression

# instantiate, train, evaluate
logreg = LogisticRegression()
logreg.fit(X_train_ss,y_train)
print(f"train acc: {logreg.score(X_train_ss,y_train)}")
print(f"test acc: {logreg.score(X_test_ss,y_test)}")

# create df to visualize coefs
logreg_scores = pd.DataFrame(columns=X.columns,data=logreg.coef_).T
# set up confusion matrix
cm = confusion_matrix(y_test,logreg.predict(X_test_ss))

# plot matrix
plot_confusion_matrix(logreg, X_test_ss, y_test, cmap='BuGn', display_labels=['no prison time', 'prison time']);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10498870154701895, 'fraction_numerical': 0.01095080827394403, 'mean_word_length': 7.013927576601671, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '501973', 'license': 'MIT', 'n_tokens_mistral': 2044, 'n_tokens_neox': 1951, 'n_words': 509, 'path': 'code/03_Modeling_logreg.ipynb', 'repo_name': '123CarlosRivera/project5', 'size': '5753'}