<jupyter_start><jupyter_text>*Import Libraries and Dataset*<jupyter_code>import sys
!pip install category_encoders

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from category_encoders import OneHotEncoder, OrdinalEncoder
from sklearn.ensemble import RandomForestClassifier
from sklearn.impute import SimpleImputer
from sklearn.metrics import plot_confusion_matrix, classification_report
from sklearn.metrics import roc_curve, roc_auc_score


cols = ['ID', 'Age', 'Gender', 'Education', 'Country', 'Ethnicity', 'NScore_Neuroticism',
        'EScore_Extraversion', 'OScore_Open_to_Exp', 'AScore_Agreeableness',
        'CScore_Conscientiousness', 'Impulsiveness', 'Sensation_Seeing', 'Alcohol',
        'Amphetamine', 'Amyl', 'Benzos', 'Caffeine', 'Cannabis', 'Chocolate', 'Cocaine',
        'Crack', 'Ecstasy', 'Heroin', 'Ketamine', 'Legal_Highs', 'LSD', 'Meth', 
        'Mushrooms', 'Nicotine', 'Semeron', 'VSA']
df = pd.read_csv('/content/drug_consumption.data',
                 header = 0, names= cols, index_col = 'ID',
                 dtype= 'object')

df = df.copy()<jupyter_output>Requirement already satisfied: category_encoders in /usr/local/lib/python3.7/dist-packages (2.2.2)
Requirement already satisfied: pandas>=0.21.1 in /usr/local/lib/python3.7/dist-packages (from category_encoders) (1.1.5)
Requirement already satisfied: scipy>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from category_encoders) (1.4.1)
Requirement already satisfied: numpy>=1.14.0 in /usr/local/lib/python3.7/dist-packages (from category_encoders) (1.19.5)
Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.7/dist-packages (from category_encoders) (0.22.2.post1)
Requirement already satisfied: statsmodels>=0.9.0 in /usr/local/lib/python3.7/dist-packages (from category_encoders) (0.10.2)
Requirement already satisfied: patsy>=0.5.1 in /usr/local/lib/python3.7/dist-packages (from category_encoders) (0.5.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.21.1->category_encoders) (2.8.1)
Requirement alre[...]<jupyter_text>*Wrangle and Clean Data*<jupyter_code>def wrangle(df):
  cutoff = 500
  hcc_cols = [col for col in df.select_dtypes('object').columns
              if df[col].nunique() > cutoff]
  df.drop(columns=hcc_cols, inplace=True)
  single_cols = [col for col in df.select_dtypes('object').columns
              if df[col].nunique() == 1]
  df.drop(columns= single_cols, inplace= True)
  return df

  df = wrangle(df)
df.info()
df.head()
df['Age'] = df['Age'].str.replace('-0.95197', '18-24').str.replace('-0.07854', '25-34').str.replace('0.49788', '35-44').str.replace('1.09449', '45-54').str.replace('1.82213', '55-64').str.replace('2.59171', '65+')
df['Gender'] = df['Gender'].str.replace('-0.48246', 'Female').str.replace('0.48246', 'Male')
df['Cannabis'] = df['Cannabis'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Education'] = df['Education'].str.replace('-2.43591', 'Left_before_16').str.replace('-1.73790', 'Left_at_16').str.replace('-1.43719', 'Left_at_17').str.replace('-1.22751', 'Left_at_18').str.replace('-0.61113', 'Some_College').str.replace('-0.05921', 'Professional_Cert').str.replace('0.45468', 'Univ_degree').str.replace('1.16365', 'Masters_Degree').str.replace('1.98437', 'Dr_Degree')
df['Ethnicity'] = df['Ethnicity'].str.replace('-0.50212', 'Asian').str.replace('-1.10702', 'Black').str.replace('1.90725', 'Bl_As').str.replace('0.12600', 'Wh_As').str.replace('-0.22166', 'Wh_Bl').str.replace('0.11440', 'Other').str.replace('-0.31685', 'White')
df['Country'] = df['Country'].str.replace('-0.09765', 'Australia').str.replace('0.24923', 'Canada').str.replace('-0.46841', 'New_Zealand').str.replace('-0.28519', 'Other').str.replace('0.21128', 'Ireland').str.replace('0.96082', 'UK').str.replace('-0.57009', 'USA')
df['Alcohol'] = df['Alcohol'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Amphetamine'] = df['Amphetamine'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Amyl'] = df['Amyl'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Benzos'] = df['Benzos'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Caffeine'] = df['Caffeine'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Chocolate'] = df['Chocolate'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Cocaine'] = df['Cocaine'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Crack'] = df['Crack'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Ecstasy'] = df['Ecstasy'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Heroin'] = df['Heroin'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Ketamine'] = df['Ketamine'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Legal_Highs'] = df['Legal_Highs'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['LSD'] = df['LSD'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Meth'] = df['Meth'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Mushrooms'] = df['Mushrooms'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Nicotine'] = df['Nicotine'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['Semeron'] = df['Semeron'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['VSA'] = df['VSA'].str.replace('CL6', 'Used_24hrs').str.replace('CL5', 'Used_7days').str.replace('CL4', 'Used_Month').str.replace('CL3', 'Used_Year').str.replace('CL2', 'Used_Decade').str.replace('CL1', 'Used_Life').str.replace('CL0', 'Never_Used')
df['NScore_Neuroticism'] = df['NScore_Neuroticism'].astype(float)
df['EScore_Extraversion'] = df['EScore_Extraversion'].astype(float)
df['OScore_Open_to_Exp'] = df['OScore_Open_to_Exp'].astype(float)
df['AScore_Agreeableness'] = df['AScore_Agreeableness'].astype(float)
df['CScore_Conscientiousness'] = df['CScore_Conscientiousness'].astype(float)
df['Impulsiveness'] = df['Impulsiveness'].astype(float)
df['Sensation_Seeing'] = df['Sensation_Seeing'].astype(float)<jupyter_output><empty_output><jupyter_text>*Explore Data*<jupyter_code>df['Age'].value_counts(normalize= True)
df['OScore_Open_to_Exp'].value_counts()
df.head()<jupyter_output><empty_output><jupyter_text>*Choose target and split data*<jupyter_code>target = 'Meth'
X = df.drop(columns= target)
y = df[target]
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size= 0.2, random_state= 42)

print(X_train.shape)
print(y_train.shape)
print(X_val.shape)
print(y_val.shape)<jupyter_output>(1507, 30)
(1507,)
(377, 30)
(377,)<jupyter_text>*Build Models*<jupyter_code>model_rf = make_pipeline(OrdinalEncoder(),
                         SimpleImputer(),
                         RandomForestClassifier(max_depth= 10, n_jobs= -1))

model_rf.fit(X_train, y_train)
print('Validation Score: ', model_rf.score(X_val, y_val))
print('Baseline accuracy ', y_train.value_counts(normalize= True).max())
df_numerical = df.select_dtypes(exclude= 'object')
df_numerical.head()
from xgboost import XGBClassifier

model_boost = make_pipeline(OrdinalEncoder(),
                            SimpleImputer(strategy= 'median'),
                            XGBClassifier(n_estimators= 75, learning_rate= 0.2,
                                          random_state= 42))
model_boost.fit(X_train, y_train);
target= 'Amphetamine'
X2 = df_cat.drop(columns= target)
y2 = df_cat[target]

X2_train, X2_val, y2_train, y2_val = train_test_split(X2, y2, test_size= 0.3, 
                                                      random_state= 42)
model_boost_cat = make_pipeline(OrdinalEncoder(),
                            SimpleImputer(strategy= 'median'),
                            XGBClassifier(n_estimators= 50, learning_rate= 0.1,
                                          random_state= 42, n_jobs= -1))
model_boost_cat.fit(X2_train, y2_train);<jupyter_output><empty_output><jupyter_text>*Check Metrics and Visualize Data*<jupyter_code>print('Training Accuracy: ', model_boost.score(X_train, y_train))
print('Validation Accuracy: ', model_boost.score(X_val, y_val))
baseline_acc2 = y2_train.value_counts(normalize= True).max()
print('Baseline Accuracy 2: ', baseline_acc2)
print('Training Accuracy 2: ', model_boost_cat.score(X2_train, y2_train))
print('Validation Accuracy 2: ', model_boost_cat.score(X2_val, y2_val))
df_cat = df.select_dtypes(exclude= 'number')
df_cat.head()
rf = model_rf.named_steps['randomforestclassifier']
importances = pd.Series(rf.feature_importances_, X_train.columns)

n = 10
plt.figure(figsize=(10,n/2))
plt.title(f'Top {n} features')
importances.sort_values()[-n:].plot.barh(color='grey')
plt.xlabel('Feature Importance for Meth')
plt.ylabel('Features');
boost_cat = model_boost_cat.named_steps['xgbclassifier']
importances_cat = pd.Series(boost_cat.feature_importances_, X2_val.columns)

n = 10
plt.figure(figsize=(10,n/2))
plt.title(f'Top {n} features')
importances_cat.sort_values()[-n:].plot.barh(color= 'blue')
plt.xlabel('Feature Importance with relation to Amphetamine')
plt.ylabel('Features');
df_numerical.shape
heroin = df_cat['Heroin']
cannabis = df_cat['Cannabis']
meth = df_cat['Meth']
alcohol = df_cat['Alcohol']
ketamine = df_cat['Ketamine']
acid = df_cat['LSD']
mushrooms = df_cat['Mushrooms']
nicotine = df_cat['Nicotine']
caffeine = df_cat['Caffeine']
chocolate = df_cat['Chocolate']

age_group = df_cat['Age']
O_score = df_numerical['OScore_Open_to_Exp']
C_score = df_numerical['CScore_Conscientiousness']
E_score = df_numerical['EScore_Extraversion']
A_score = df_numerical['AScore_Agreeableness']
N_score = df_numerical['NScore_Neuroticism']
impulsiveness = df_numerical['Impulsiveness']
sen_seeing = df_numerical['Sensation_Seeing']
education = df_cat['Education']

illegal_drugs = pd.DataFrame(data= df_cat, columns= ['Heroin', 'Meth', 'Ketamine',
                                                     'LSD', 'Mushrooms'])
legal_drugs = pd.DataFrame(data= df_cat, columns= ['Cannabis', 'Nicotine', 'Caffeine',
                                                   'Alcohol', 'Chocolate'])
import seaborn as sns

sns.catplot(data= df_cat, x= O_score, y= acid);
sns.catplot(data= df_cat, x= O_score, y= cannabis);<jupyter_output><empty_output><jupyter_text>**O-Score(Openness)** is a person's willingness to be open to new ideas and experiences.  Scoring is again, very similar to the the E-Score in the sense that a higher positive score indicates you are more likely to experiment with various substances. **C-Score(Conscientiousness)** shows organization vs chaos. At the highest end of the scale, perfectionism and OCD are prevalent.  On the lowest end of the scale would be those who act before thinking and spontaneous individuals.  **E-Score(Extraversion)** in this scenario uses 0 as a start point and a positive score indicating you are more extroverted while a negative score indicates a personality that is more introverted. **A-Score(Agreeableness)** is how agreeable or prone to persuasion an individual is. Someone with a higher score is more likely to be a people pleaser, while someone with a low score is more likely to be invested in personal interests. **N-Score(Neuroticism)** is scored on a scale of 0-12. A score of 0-6 means there is a general lack of Neuroticism present and 6-12 indicates a positive result for Neuroticism.  In this case, we will be using a scale from 0.0 to 1.2 with 0.6 being the cutoff.<jupyter_code>clr = classification_report(y2_val, model_boost_cat.predict(X2_val))
print(clr)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.19162771794350741, 'fraction_numerical': 0.03881324933956513, 'mean_word_length': 6.081055155875299, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '129409', 'license': 'MIT', 'n_tokens_mistral': 5658, 'n_tokens_neox': 5327, 'n_words': 864, 'path': 'Final_4/1/2021_Andrew_Haney_Unit2_Project_Pred_Model.ipynb', 'repo_name': 'Andrew-Haney/DS-Unit-2-Linear-Models', 'size': '14763'}