<jupyter_start><jupyter_text>Lambda School Data Science*Unit 2, Sprint 3, Module 3*--- Permutation & BoostingYou will use your portfolio project dataset for all assignments this sprint. AssignmentComplete these tasks for your project, and document your work.- [ ] If you haven't completed assignment 1, please do so first.- [ ] Continue to clean and explore your data. Make exploratory visualizations.- [ ] Fit a model. Does it beat your baseline? - [ ] Try xgboost.- [ ] Get your model's permutation importances.You should try to complete an initial model today, because the rest of the week, we're making model interpretation visualizations.But, if you aren't ready to try xgboost and permutation importances with your dataset today, that's okay. You can practice with another dataset instead. You may choose any dataset you've worked with previously.The data subdirectory includes the Titanic dataset for classification and the NYC apartments dataset for regression. You may want to choose one of these datasets, because example solutions will be available for each. ReadingTop recommendations in _**bold italic:**_ Permutation Importances- _**[Kaggle / <NAME>: Machine Learning Explainability](https://www.kaggle.com/dansbecker/permutation-importance)**_- [<NAME>: Interpretable Machine Learning](https://christophm.github.io/interpretable-ml-book/feature-importance.html) (Default) Feature Importances  - [Ando Saabas: Selecting good features, Part 3, Random Forests](https://blog.datadive.net/selecting-good-features-part-iii-random-forests/)  - [Ter<NAME>, et al: Beware Default Random Forest Importances](https://explained.ai/rf-importance/index.html) Gradient Boosting  - [A Gentle Introduction to the Gradient Boosting Algorithm for Machine Learning](https://machinelearningmastery.com/gentle-introduction-gradient-boosting-algorithm-machine-learning/)  - [An Introduction to Statistical Learning](http://www-bcf.usc.edu/~gareth/ISL/ISLR%20Seventh%20Printing.pdf), Chapter 8  - _**[Gradient Boosting Explained](https://www.gormanalysis.com/blog/gradient-boosting-explained/)**_ — <NAME>  - [Gradient Boosting Explained](http://arogozhnikov.github.io/2016/06/24/gradient_boosting_explained.html) — <NAME>  - [How to explain gradient boosting](https://explained.ai/gradient-boosting/) — <NAME> & <NAME><jupyter_code>from google.colab import drive
drive.mount('/content/drive')
import pandas as pd<jupyter_output><empty_output><jupyter_text>Continue to clean and explore your data. Make exploratory visualizations.<jupyter_code>#See LS_DS_232_assignment
df = pd.read_csv('/content/drive/My Drive/Lambda/Unit 2/Basketball Data/eng_shots_log.csv')
df = df.drop(columns='Unnamed: 0')
df.head(1)<jupyter_output><empty_output><jupyter_text>New feature: previous shot made<jupyter_code>df = df.drop(columns=['PREV_MADE'])
df['PREV_SHOT'] = 'first_shot'
df.head(15) 
trial = df.head(15)
player1 = ''
player2 = ''

for index, x in enumerate(trial['SHOT_RESULT']):
    #account for the first observation 
  if (index == 0):
    trial.loc[index, 'PREV_SHOT'] = 'first_shot'
    player2 = trial.loc[index, 'player_name']
    print('yep')

  if (index >= 1):
      player1 = trial.loc[index, 'player_name']
      if player1 == player2:
        #check to see if this shot was in the same game as the previous
        prev_game = trial.loc[(index-1), 'GAME_ID']
        this_game = trial.loc[index, 'GAME_ID']

        if (prev_game == this_game):
          #set the prev_shot to the player's previous shot
          trial.loc[index, 'PREV_SHOT'] = trial.loc[(index-1), 'SHOT_RESULT']
        else:
          #set the prev_shot to 'first_shot'
          trial.loc[index, 'PREV_SHOT'] = 'first_shot'
      


trial.head(15)
#now do it on the df
player1 = ''
player2 = ''

for index, x in enumerate(df['SHOT_RESULT']):
    #account for the first observation 
  if (index == 0):
    df.loc[index, 'PREV_SHOT'] = 'first_shot'
    player2 = df.loc[index, 'player_name']
    print('yep')

  if (index >= 1):
      player1 = df.loc[index, 'player_name']
      if player1 == player2:
        #check to see if this shot was in the same game as the previous
        prev_game = df.loc[(index-1), 'GAME_ID']
        this_game = df.loc[index, 'GAME_ID']

        if (prev_game == this_game):
          #set the prev_shot to the player's previous shot
          df.loc[index, 'PREV_SHOT'] = df.loc[(index-1), 'SHOT_RESULT']
          #set player2 to the current player
          player2 = df.loc[index, 'player_name']
        else:
          #set the prev_shot to 'first_shot'
          df.loc[index, 'PREV_SHOT'] = 'first_shot'
          print('first')
      else:
        df.loc[index, 'PREV_SHOT'] = 'first_shot'
        player2 = df.loc[index, 'player_name']
        print('new player')
      
df.tail(20)<jupyter_output><empty_output><jupyter_text>Fit a model. Does it beat your baseline?<jupyter_code>#Again, see LS_DS_232_assignment:
#baseline accuracy: 54.8%
#model 1 accuracy: 60.5%
#model 2 accuracy: 60.9%

target = 'SHOT_RESULT'
y = df[target]
y.value_counts(normalize=True)
#Can you make a fast, first model that beats guessing?
import numpy as np
from sklearn.model_selection import train_test_split

train = df

# Split df into into train & test sets
train, test = train_test_split(train, train_size=0.80, test_size=0.20, 
                              stratify=train['SHOT_RESULT'], random_state=42)
train.shape, test.shape
#Split df into train and validation sets
train, val = train_test_split(train, train_size=0.80, test_size=0.20, 
                              stratify=train['SHOT_RESULT'], random_state=42)
train.shape, val.shape
# Get a dataframe with all train columns except the target
train_features = train.drop(columns=[target])

# Get a list of the numeric features
numeric_features = train_features.select_dtypes(include='number').columns.tolist()

# Get a series with the cardinality of the nonnumeric features
cardinality = train_features.select_dtypes(exclude='number').nunique()

# Get a list of all categorical features with cardinality <= 50
categorical_features = cardinality[cardinality <= 50].index.tolist()

# Combine the lists 
features = numeric_features + categorical_features
# Arrange data into X features matrix and y target vector 
X_train = train[features]
y_train = train[target]
X_val = val[features]
y_val = val[target]
X_test = test[features]
y_test = test[target]
#install category encoders
!pip install category_encoders
#Fit a model using a pipeline

import category_encoders as ce
from sklearn.impute import SimpleImputer
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import make_pipeline

pipeline = make_pipeline(
    ce.one_hot.OneHotEncoder(),
    SimpleImputer(),
    RandomForestClassifier(n_estimators=100, n_jobs= -1, random_state=42)
)

pipeline.fit(X_train, y_train)
print(f'Valdiation accuracy: {pipeline.score(X_val, y_val)}')<jupyter_output>Valdiation accuracy: 0.6084134498072324<jupyter_text>Get your model's permutation importances<jupyter_code>%%capture
import sys

if 'google.colab' in sys.modules:
    DATA_PATH = 'https://raw.githubusercontent.com/LambdaSchool/DS-Unit-2-Applied-Modeling/master/data/'
    !pip install category_encoders==2.*
    !pip install eli5

import eli5
from eli5.sklearn import PermutationImportance

transformers = make_pipeline(
    ce.ordinal.OrdinalEncoder(),
    SimpleImputer()
)

X_train_transformed = transformers.fit_transform(X_train)
X_val_transformed = transformers.transform(X_val)

model = RandomForestClassifier(n_estimators=100, random_state=42, n_jobs=-1)
model.fit(X_train_transformed, y_train)
permuter = PermutationImportance(
    model,
    scoring = 'accuracy',
    n_iter=5,
    random_state=42
)

permuter.fit(X_val_transformed, y_val)
feature_names = X_val.columns.to_list()

pd.Series(permuter.feature_importances_, feature_names).sort_values(ascending=False)
permuter.feature_importances_std_
eli5.show_weights(permuter,
                  top=None, 
                  feature_names = feature_names)<jupyter_output><empty_output><jupyter_text>Try xgboost.<jupyter_code>from xgboost import XGBClassifier

pipeline = make_pipeline(
    ce.ordinal.OrdinalEncoder(),
    XGBClassifier(n_estimators= 100, random_state = 42, n_jobs = -1)
)

pipeline.fit(X_train, y_train)
from sklearn.metrics import accuracy_score
y_pred = pipeline.predict(X_val)
print(f'Val accuracy score: {accuracy_score(y_val, y_pred)}')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.108171941426547, 'fraction_numerical': 0.015706188001889467, 'mean_word_length': 4.510084580351334, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 8, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '353909', 'license': 'MIT', 'n_tokens_mistral': 2839, 'n_tokens_neox': 2676, 'n_words': 715, 'path': 'module3-permutation-boosting/LS_DS_233_assignment.ipynb', 'repo_name': 'ndow33/DS-Unit-2-Applied-Modeling', 'size': '8512'}