<jupyter_start><jupyter_text>Lambda School Data Science*Unit 2, Sprint 3, Module 1*--- Define ML problemsYou will use your portfolio project dataset for all assignments this sprint. AssignmentComplete these tasks for your project, and document your decisions.- [ ] Choose your target. Which column in your tabular dataset will you predict?- [ ] Is your problem regression or classification?- [ ] How is your target distributed?    - Classification: How many classes? Are the classes imbalanced?    - Regression: Is the target right-skewed? If so, you may want to log transform the target.- [ ] Choose your evaluation metric(s).    - Classification: Is your majority class frequency >= 50% and < 70% ? If so, you can just use accuracy if you want. Outside that range, accuracy could be misleading. What evaluation metric will you choose, in addition to or instead of accuracy?    - Regression: Will you use mean absolute error, root mean squared error, R^2, or other regression metrics?- [ ] Choose which observations you will use to train, validate, and test your model.    - Are some observations outliers? Will you exclude them?    - Will you do a random split or a time-based split?- [ ] Begin to clean and explore your data.- [ ] Begin to choose which features, if any, to exclude. Would some features "leak" future information?If you haven't found a dataset yet, do that today. [Review requirements for your portfolio project](https://lambdaschool.github.io/ds/unit2) and choose your dataset.Some students worry, ***what if my model isn't “good”?*** Then, [produce a detailed tribute to your wrongness. That is science!](https://twitter.com/nathanwpyle/status/1176860147223867393)<jupyter_code># Choose your target. Which column in your tabular dataset will you predict?:
#       -(COLUMN I CREATE CALLED 'HIGHCOST' which will have 2 different available entries: yes or no)
# Is your problem regression or classification?:
#       -CLASSIFICATION [HighCost])
# How is your target distributed?
# Classification: How many classes? Are the classes imbalanced?:
#       -(TWO; no) cutoff point between low and high cost is $1.41
# Classification: Is your majority class frequency >= 50% and < 70% ? If so, you can just use accuracy 
# if you want. Outside that range, accuracy could be misleading. What evaluation metric will you choose, 
# in addition to or instead of accuracy?:
#       -
# Choose which observations you will use to train, validate, and test your model:
#       -(AVERAGEPRICE, TOTALVOLUME, TOTALBAGS, TYPE & REGION)
# Are some observations outliers? Will you exclude them?:
#       -THERE ARE OUTLIERS/WILL INCLUDE THEM
# Will you do a random split or a time-based split? 
#       -RANDOM SPLIT
# Begin to clean and explore your data.
# Begin to choose which features, if any, to exclude. Would some features "leak" future information? 
#       -(WILL PROBABLY EXCLUDE THE 'PLU' COLUMNS & SIZE OF BAGS COLUMNS)
import pandas as pd
# Importing dataset about the average price per avocado:
df = pd.read_csv('avocado.csv')
# Examining df:
df.head()
# Getting a sample of the data:
df.sample(5)
# Looking at df's shape:
df.shape
# Check for NaN/null values:
df.isnull().sum()
# Looking for outliers:
import seaborn as sns
sns.boxplot(x=df['AveragePrice']);
# Removing outliers to check if I get different results.
# Removing any rows where value in 'AveragePrice' is >= 2.5:
# df = df[df['AveragePrice'] < 2.5]
# df.head()
# Checking spread of values to determine how to
# Make my target column
# df['AveragePrice'].describe()
# Making target column called 'HighCost',
# Then looking at sample of data:
# import numpy as np
# df['HighCost'] = np.where(df['AveragePrice'] >= 1.41, 'yes', 'no')
# df.sample(5)
# Renaming column Unnamed:0 to 'PurchaseInstance':
# df = df.rename({'Unnamed: 0': 'PurchaseInstance'}, axis=1)
# df.sample(5)
# Dropping columns for PLU, and bag size:  (adding average price as it tends to overfit)
df = df.drop(columns=['4046', '4225', '4770', 'Small Bags', 'XLarge Bags', 'Total Volume', 'Total Bags'])
df.head()
# Converting 'Date' to datetime:
df['Date'] = pd.to_datetime(df['Date'], infer_datetime_format=True)
df.head()
# Extracting 'Month' from 'Date', (since time of year may be a determining
# factor when predicting price) then dropping 'Date':
df['MonthRecorded'] = df['Date'].dt.month
df = df.drop(columns='Date')
df.head()
# Removing column called PurchaseInstance, as I don't think it's needed:
# df = df.drop(columns='PurchaseInstance')
# df.head()
# Determining the majority class:
from sklearn.model_selection import train_test_split
train, test = train_test_split(df, random_state=42)
train.shape, test.shape
target = 'type'
y_train = train[target]
y_train.value_counts(normalize=True)
# Majority Class for every prediction:
majority_class = y_train.mode()[0]
y_pred_train = [majority_class]*len(y_train)
# Getting accuracy score:
from sklearn.metrics import accuracy_score
accuracy_score(y_train, y_pred_train)
# Assignment for Sunday, April 26th 2020:
# Continue to clean and explore your data.
# For the evaluation metric you chose, what score would you get just by guessing?
# Can you make a fast, first model that beats guessing?
# Removing the dependant variable/target from test:
test = test.drop(columns='type')
train.shape, test.shape
# Splitting into train, val & test:
train, val = train_test_split(train, train_size=0.80, test_size=0.20,
                             stratify=train['type'], random_state=42)
train.shape, val.shape, test.shape
target = 'type'
y_train = train[target]
y_train.value_counts(normalize=True)
# Majority Class for every prediction a 2nd time:
majority_class = y_train.mode()[0]
y_pred_train = [majority_class]*len(y_train)
# Accuracy score a 2nd time:
accuracy_score(y_train, y_pred_train)
y_val = val[target]
y_pred = [majority_class]*len(y_val)
accuracy_score(y_val, y_pred)
# Manually selecting features:
features = ['region', 'MonthRecorded', 'year', 'Large Bags', 'AveragePrice']
# Arrange the data into features and target:
X_train = train[features]
y_train = train[target]
X_val = val[features]
y_val = val[target]
X_test = test[features]
# Imports to start the pipeline:
import category_encoders as ce
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import OneHotEncoder
pipeline = make_pipeline(
    ce.OrdinalEncoder(), 
    SimpleImputer(strategy='median'), 
    RandomForestClassifier(n_estimators=100, random_state=42, n_jobs=-1)
)

# Fit on train, score on val
pipeline.fit(X_train, y_train)
print('Validation Accuracy', pipeline.score(X_val, y_val))
pipeline.predict(X_test)
# Begin Module 3
# Permutation and xgboost:
# Getting importances:
randfor = pipeline.named_steps['randomforestclassifier']
importances = pd.Series(randfor.feature_importances_, X_train.columns)
# Plot the feature importances:
import matplotlib.pyplot as plt
n = 5
plt.figure(figsize=(10,n/2))
plt.title(f'The {n} features I used')
importances.sort_values()[-n:].plot.barh(color='red');
# Permutation importance w/ eli5:
transformers = make_pipeline(
    ce.ordinal.OrdinalEncoder(),
    SimpleImputer()
)

X_train_tranformed = transformers.fit_transform(X_train)
X_val_tranformed = transformers.transform(X_val)

model = RandomForestClassifier(n_estimators=100, random_state=42, n_jobs=-1)
model.fit(X_train_tranformed, y_train)
import eli5
from eli5.sklearn import PermutationImportance
permuter = PermutationImportance(
    model,
    scoring='accuracy',
    n_iter=10,
    random_state=42
)

permuter.fit(X_val_tranformed, y_val)
feature_names = X_val.columns.to_list()
pd.Series(permuter.feature_importances_, feature_names).sort_values(ascending=True)
permuter.feature_importances_std_
# Number of features * number of iterations:
n*10
eli5.show_weights(permuter, 
                  top=None, 
                  feature_names=feature_names)
# XGBoost:
from xgboost import XGBClassifier

pipeline = make_pipeline(
    ce.ordinal.OrdinalEncoder(),
    XGBClassifier(n_estimators=100, random_state=42, n_jobs=-1)
)

pipeline.fit(X_train, y_train)
from sklearn.metrics import accuracy_score

y_pred = pipeline.predict(X_val)
print(f'Val accuracy: {accuracy_score(y_val, y_pred)}')
# Begin assignment for module 4:
# Make at least 1 partial dependence plot to explain your model.
# Make at least 1 Shapley force plot to explain an individual prediction.
# Share at least 1 visualization (of any type) on Slack!
# I am still working on how to LabelEncode or OHE my columns in order for
# The next several cells to work.
from sklearn.metrics import r2_score
from xgboost import XGBClassifier

gb = make_pipeline(
    ce.OrdinalEncoder
(), 
    XGBClassifier(n_estimators=200, objective='reg:squarederror', n_jobs=-1)
)

gb.fit(X_train, y_train)
y_pred = gb.predict(X_val)
print('Gradient Boosting R^2', r2_score(y_val, y_pred))
# I can do the next cells once the above are corrected:
# Partial Dependence Plots w/ 1 feature:
import matplotlib.pyplot as plt
plt.rcParams['figure.dpi'] = 72
from pdpbox.pdp import pdp_isolate, pdp_plot

depfeature = 'AveragePrice'

isolated = pdp_isolate(
    model=gb,
    dataset=X_val,
    model_features=X_val.columns,
    feature=depfeature
)
pdp_plot(isolated, feature_name=depfeature);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09843518714315923, 'fraction_numerical': 0.013533516599703954, 'mean_word_length': 5.059577194106342, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 9, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '326950', 'license': 'MIT', 'n_tokens_mistral': 3062, 'n_tokens_neox': 2909, 'n_words': 1013, 'path': 'module1-define-ml-problems/Avocado_Data_Set.ipynb', 'repo_name': 'CurdtMillion/DS-Unit-2-Applied-Modeling', 'size': '9458'}