<jupyter_start><jupyter_text>Starbucks Capstone Challenge IntroductionThis data set contains simulated data that mimics customer behavior on the Starbucks rewards mobile app. Once every few days, Starbucks sends out an offer to users of the mobile app. An offer can be merely an advertisement for a drink or an actual offer such as a discount or BOGO (buy one get one free). Some users might not receive any offer during certain weeks. Not all users receive the same offer, and that is the challenge to solve with this data set.Your task is to combine transaction, demographic and offer data to determine which demographic groups respond best to which offer type. This data set is a simplified version of the real Starbucks app because the underlying simulator only has one product whereas Starbucks actually sells dozens of products.Every offer has a validity period before the offer expires. As an example, a BOGO offer might be valid for only 5 days. You'll see in the data set that informational offers have a validity period even though these ads are merely providing information about a product; for example, if an informational offer has 7 days of validity, you can assume the customer is feeling the influence of the offer for 7 days after receiving the advertisement.You'll be given transactional data showing user purchases made on the app including the timestamp of purchase and the amount of money spent on a purchase. This transactional data also has a record for each offer that a user receives as well as a record for when a user actually views the offer. There are also records for when a user completes an offer. Keep in mind as well that someone using the app might make a purchase through the app without having received an offer or seen an offer. ExampleTo give an example, a user could receive a discount offer buy 10 dollars get 2 off on Monday. The offer is valid for 10 days from receipt. If the customer accumulates at least 10 dollars in purchases during the validity period, the customer completes the offer.However, there are a few things to watch out for in this data set. Customers do not opt into the offers that they receive; in other words, a user can receive an offer, never actually view the offer, and still complete the offer. For example, a user might receive the "buy 10 dollars get 2 dollars off offer", but the user never opens the offer during the 10 day validity period. The customer spends 15 dollars during those ten days. There will be an offer completion record in the data set; however, the customer was not influenced by the offer because the customer never viewed the offer. CleaningThis makes data cleaning especially important and tricky.You'll also want to take into account that some demographic groups will make purchases even if they don't receive an offer. From a business perspective, if a customer is going to make a 10 dollar purchase without an offer anyway, you wouldn't want to send a buy 10 dollars get 2 dollars off offer. You'll want to try to assess what a certain demographic group will buy when not receiving any offers. Final AdviceBecause this is a capstone project, you are free to analyze the data any way you see fit. For example, you could build a machine learning model that predicts how much someone will spend based on demographics and offer type. Or you could build a model that predicts whether or not someone will respond to an offer. Or, you don't need to build a machine learning model at all. You could develop a set of heuristics that determine what offer you should send to each customer (i.e., 75 percent of women customers who were 35 years old responded to offer A vs 40 percent from the same demographic to offer B, so send offer A).  Data SetsThe data is contained in three files:* portfolio.json - containing offer ids and meta data about each offer (duration, type, etc.)* profile.json - demographic data for each customer* transcript.json - records for transactions, offers received, offers viewed, and offers completedHere is the schema and explanation of each variable in the files:**portfolio.json*** id (string) - offer id* offer_type (string) - type of offer ie BOGO, discount, informational* difficulty (int) - minimum required spend to complete an offer* reward (int) - reward given for completing an offer* duration (int) - time for offer to be open, in days* channels (list of strings)**profile.json*** age (int) - age of the customer * became_member_on (int) - date when customer created an app account* gender (str) - gender of the customer (note some entries contain 'O' for other rather than M or F)* id (str) - customer id* income (float) - customer's income**transcript.json*** event (str) - record description (ie transaction, offer received, offer viewed, etc.)* person (str) - customer id* time (int) - time in hours since start of test. The data begins at time t=0* value - (dict of strings) - either an offer id or transaction amount depending on the record<jupyter_code># Import libraries 
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from scipy import stats
import math
import json
import datetime
import locale
% matplotlib inline

# Import machine learning related libraries
from sklearn.metrics import accuracy_score, f1_score
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier

# read in the json files
portfolio = pd.read_json('data/portfolio.json', orient='records', lines=True)
profile = pd.read_json('data/profile.json', orient='records', lines=True)
transcript = pd.read_json('data/transcript.json', orient='records', lines=True)<jupyter_output><empty_output><jupyter_text>Review Data **Portfolio*** **Offers sent during 30-day test period (10 offers x 6 fields)*** id (string) - offer id* offer_type (string) - type of offer ie BOGO, discount, informational* difficulty (int) - minimum required spend to complete an offer* reward (int) - reward given for completing an offer* duration (int) - time for offer to be open, in days* channels (list of strings)<jupyter_code># Read the portfolio samples
print("Portfolio has %d rows and %d columns."  %portfolio.shape)

portfolio.head()
# Count the number of different offer type
portfolio.groupby('offer_type')['id'].count()<jupyter_output><empty_output><jupyter_text>**There are 10 different offers.*** BOGO has 4 different offers* Discount has 2 different offers* Informational has 2 different offers  **Profile*** **Rewards program users (17000 users x 5 fields)*** age (int) - age of the customer * became_member_on (int) - date when customer created an app account* gender (str) - gender of the customer (note some entries contain 'O' for other rather than M or F)* id (str) - customer id* income (float) - customer's income<jupyter_code># Read the profile samples
print("Profile has %d rows and %d columns."  %profile.shape)
profile.head()
# Check null values in dataset
profile.isnull().sum()
# Identify customers at age 118 and null in gender are the same group of people.
null_gender = profile[profile['gender'].isnull()]
null_gender.age.value_counts()<jupyter_output><empty_output><jupyter_text>**Customers with age 118 also don't have value on gender and income and can be removed from the dataset.**<jupyter_code># Analyze the gender distribution for customers

# Counts the gender in Male, Female, and Others (excluding Null value)
value_counts = profile['gender'].value_counts().reset_index()
#print(value_counts) #Male - 8484, Female - 6129, Other - 212
#print(value_counts.sum()) # Total is 14825
plt.subplots(figsize=(5,5))
plt.title('Percentage Distribution for Gender',size = 20)
plt.pie(x=value_counts['gender'], labels= ['Male - 8484','Female - 6129','Other - 212'], autopct='%0.2f%%')

plt.show()<jupyter_output><empty_output><jupyter_text>**There are total 14825 valid user in this dataset (Excluded 2175 customers whose ages at 118.)*** Male - 8484 (57.23%)* Female - 6129 (41.34%)* Others - 212 (1.43%)<jupyter_code># Analyze the age distribution for customers
sorted_age = profile[profile.age < 118]
fig= plt.subplots(figsize=(6,4))
sns.distplot(sorted_age['age'], kde = False, hist_kws={'edgecolor':'blue'})
plt.title('Age Distribution of Users',size = 20)
plt.xlabel('Age of User', size =10)
plt.ylabel('Number of User', size =10)

plt.show()<jupyter_output>/home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/matplotlib/axes/_axes.py:6521: MatplotlibDeprecationWarning: 
The 'normed' kwarg was deprecated in Matplotlib 2.1 and will be removed in 3.1. Use 'density' instead.
  alternative="'density'", removal="3.1")<jupyter_text>**Most registered customers are from ages 50 to 70.*** People between age 20-40 are not the largest group of customer.<jupyter_code># Analyze the income distribution for customers
sorted_age = profile[profile.age < 118]
fig= plt.subplots(figsize=(6,4))
sns.distplot(sorted_age['income'], kde = False, hist_kws={'edgecolor':'blue'})
plt.title('Income Distribution of Users',size = 20)
plt.xlabel('Income of User', size =10)
plt.ylabel('Number of User', size =10)

plt.show()<jupyter_output>/home/ec2-user/anaconda3/envs/mxnet_p36/lib/python3.6/site-packages/matplotlib/axes/_axes.py:6521: MatplotlibDeprecationWarning: 
The 'normed' kwarg was deprecated in Matplotlib 2.1 and will be removed in 3.1. Use 'density' instead.
  alternative="'density'", removal="3.1")<jupyter_text>**Most registered customers' income are between 50K to 75K*** People earn more not necessary drinks more Starbucks.* On the other hand, people earn less money also drinks a lot of Starbucks.  **Transcript*** **Event log (306648 events x 4 fields)*** event (str) - record description (ie transaction, offer received, offer viewed, etc.)* person (str) - customer id* time (int) - time in hours since start of test. The data begins at time t=0* value - (dict of strings) - either an offer id or transaction amount depending on the record    * offer id: (string/hash) not associated with any "transaction"    * amount: (numeric) money spent in "transaction"    * reward: (numeric) money gained from "offer completed"<jupyter_code># Read the transcript samples
print("Transcript has %d rows and %d columns."  %transcript.shape)
transcript.head()
# Check any null event in the dataset
transcript.isnull().sum()<jupyter_output><empty_output><jupyter_text>**There are no null event in this transcript, but the number of event is 306534 which is different than the one provide from Udacity classroom 306534.**<jupyter_code># Count each event number
transcript['event'].value_counts().reset_index()
# Analyze the event percentage for transcript
value_counts = transcript['event'].value_counts().reset_index()
plt.subplots(figsize=(5,5))
plt.title('Percentage Distribution for Event',size = 20)
plt.pie(x=value_counts['event'], labels= value_counts['index'], autopct='%0.2f%%')
plt.show()<jupyter_output><empty_output><jupyter_text>Cleaning the dataset  First we will update the Transcript data<jupyter_code># Rename the column 'person' into 'customer_id' to match all dataset
transcript.rename(columns={'person': 'customer_id'}, inplace=True)

# Change the 'time' value from hour to day
transcript['time'] /= 24.0

# Create two different columns for the 'value' column
# 'offer id' and 'offer_id' should combine into one group and we create a column named 'offer_id'
# 'amount' contain a number that's different to offer_id so we create another column named 'amount'
    
transcript['offer_id'] = transcript.value.apply(lambda x: list(x.values())[0] if (list(x.keys())[0] in ['offer id', 'offer_id']) else np.NaN)
transcript['amount'] = transcript.value.apply(lambda x: list(x.values())[0] if (list(x.keys())[0]=='amount') else np.NaN)

# Drop the 'value' column to avoid duplication
transcript.drop(columns=['value'], inplace=True)

# Check and drop the duplicates values
print("There are %d duplicates values. Dropped successfully!"  %transcript.duplicated().sum())
transcript.drop_duplicates(inplace=True)
transcript.head()<jupyter_output><empty_output><jupyter_text>Then we update the Profile data<jupyter_code># Remove all the age 118 customers since they don't have gender and income as well
profile.drop(profile.loc[profile['age']==118].index, inplace=True)

# Change the date format into YYYY-MM-DD and split it into three columns as well
profile['member_year'] = profile.became_member_on.apply(lambda x: int(str(x)[:4]))
profile['member_month'] = profile.became_member_on.apply(lambda x: int(str(x)[4:6]))
profile['member_day'] = profile.became_member_on.apply(lambda x: int(str(x)[6:]))
profile['member_date'] = profile.became_member_on.apply(lambda x: datetime.datetime.strptime(str(x), '%Y%m%d'))

# Drop the 'became_member_on' column since we have updated version
profile.drop('became_member_on', axis=1, inplace=True)

# Change the column name of id to customer_id to avoid duplication
profile.rename(columns={'id': 'customer_id'}, inplace=True)

# Split gender into individual column
profile['male'] = profile.gender.apply(lambda x: 1 if x =='M' else 0)
profile['female'] = profile.gender.apply(lambda x: 1 if x =='F' else 0)
profile['others'] = profile.gender.apply(lambda x: 1 if x =='O' else 0)
profile.head()<jupyter_output><empty_output><jupyter_text>Finally we update the Portfolio data<jupyter_code># Rename column 'id' to 'offer_id' to make it more consistence
portfolio.rename(columns={'id': 'offer_id'}, inplace=True)

# Replace the channels column into each type of channel
channel_list = np.unique(portfolio['channels'])
for channel in channel_list[3]:
    portfolio[channel] = portfolio['channels'].apply(lambda x: 1 if channel in x else 0)
portfolio.drop('channels', axis=1, inplace=True)

# Replace the offer_type column into each offer type 
for offer in portfolio.offer_type.unique():
    portfolio[offer] = portfolio['offer_type'].apply(lambda x: 1 if offer in x else 0)
portfolio.drop('offer_type', axis=1, inplace=True)
portfolio.head()<jupyter_output><empty_output><jupyter_text>Preprocessing the data<jupyter_code># Create a dataframe that only shows the transaction
transaction_df = transcript[transcript.event=='transaction'].copy()
transaction_df.drop(columns=['offer_id', 'event'], inplace=True)
transaction_df.head()
# Create a dataframe that only shows the offer and without the transaction
offers_df = transcript[transcript.event!='transaction'].copy()

# Change the event into numerical format and create column for each of event
offers_df['received'] = offers_df.event.apply(lambda x: 1 if x == 'offer received' else 0)
offers_df['completed'] = offers_df.event.apply(lambda x: 1 if x == 'offer completed' else 0)
offers_df['viewed'] = offers_df.event.apply(lambda x: 1 if x == 'offer viewed' else 0)

# Drop columns 'event' and 'amount' for cleaner look
offers_df.drop(columns=['event', 'amount'], inplace=True)
offers_df.head()
# Plot a boxplot to see the age distrubution
plt.figure(figsize=(8, 3))
sns.boxplot(profile['age'])
plt.title('Customer Age Boxplot')
plt.xlabel('Age')
plt.show();
# Calculate the mean of this dataset and use it as a reference to plot a bar chart
mean_income = profile.income.mean()
income_by_gender = profile.groupby('gender', as_index=False).agg({'income':'mean'})
income_by_gender.plot(kind = 'bar', x = 'gender', y = 'income', legend=False, figsize=(5,5))
plt.hlines(mean_income, -1 , 3)
plt.ylabel('Income')
plt.xlabel('Gender')
plt.title('Average Income by Gender')
plt.text(2.52, mean_income, 'Average Income')
plt.show()

# Show the average income for this dataset
locale.setlocale( locale.LC_ALL, '' )
print ("Average income is %s in this dataset." %locale.currency(mean_income, grouping=True ))

# Drop gender column for better calculation
profile.drop('gender', axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Modelling Combine all these preprocessing data into one big dataset **Tips: This code block below takes about 10 minutes to run.**<jupyter_code># Initiate  a list to store the data. 
data = []

# Create a list of unique user id to iteration.
ids = offers_df['customer_id'].unique()

# Retrieve all the transaction and offer related details from the dataset for futhuer usage
for i in ids:
    customer_id = i
    customer = profile[profile['customer_id']==customer_id]
    customer_offer = offers_df[offers_df['customer_id']==customer_id]
    customer_transaction = transaction_df[transaction_df['customer_id']==customer_id]
    
    offer_received = customer_offer[customer_offer['received'] == 1]
    offer_completed = customer_offer[customer_offer['completed'] == 1]
    offer_viewed = customer_offer[customer_offer['viewed'] == 1]
    
    rows = []
    
    # Identify the is the offer successful and if there is transaction between the offer duration
    for i in range(offer_received.shape[0]):
        offer_id = offer_received.iloc[i]['offer_id']
        offer_row = portfolio.loc[portfolio['offer_id'] == offer_id]
        duration_time = offer_row['duration'].values[0]
        start_time = offer_received.iloc[i]['time']
        end_time = start_time + duration_time
        
        offer_completed_check = np.logical_and(offer_completed['time'] >= start_time, offer_completed['time'] <= end_time)
        offer_viewed_check = np.logical_and(offer_viewed['time'] >= start_time, offer_viewed['time'] <=end_time)
        offer_successful = offer_completed_check.sum() > 0 and offer_viewed_check.sum() > 0
        
        transaction_check = np.logical_and(customer_transaction['time'] >= start_time, customer_transaction['time'] <= end_time)
        transaction_data = customer_transaction[transaction_check]
        transaction_total_amount = transaction_data['amount'].sum()
        
        row = {
                'offer_id': offer_id,
                'customer_id': customer_id,
                'time': start_time,
                'total_amount': transaction_total_amount,
                'offer_successful': int(offer_successful),
            }
        row.update(offer_row.iloc[0,0:].to_dict())
        rows.append(row)
    data.extend(rows)
    
# Creates dataframe for the data
data = pd.DataFrame(data)
# Merge the data to the profile to have a final dataset for 
final = data.merge(profile, how = 'inner', on = 'customer_id')
final.head()
# Print all the columns header from the final dataset
list(final)<jupyter_output><empty_output><jupyter_text>Because we want to focus on the demographic characteristic and the offer feature. So we will drop some of the column that might not help make this model more acurrate<jupyter_code># Drop some of the columns that might not needed for the Machine Learning
final.drop(columns=['customer_id','offer_id','member_day','member_month','member_date','member_year','email','social','web','mobile'], inplace=True)
final.head(1)
target = final['offer_successful']
features = final.drop(columns=['offer_successful'])
from sklearn.preprocessing import MinMaxScaler

# Initialize a scaler
scaler = MinMaxScaler()
features_scaled = pd.DataFrame(scaler.fit_transform(features.astype(float)))
features_scaled.columns = features.columns
features_scaled.index = features.index
# Print it out to test it's working or not
features_scaled.head()
X_train, X_test, y_train, y_test = train_test_split(features_scaled.values, target.values, test_size=0.3, random_state=42)
X_train
# Rewrite out x_train and x_test because of there are itergers/floats in our columns
X_train = X_train[:, 2:].astype('float64')
X_test = X_test[:, 2:].astype('float64')

# Convert train and test labels to array
y_train = y_train.ravel()
y_test = y_test.ravel()

# Print out the number of training set and testing set for reference
print("Training set has %d" %(X_train.shape[0]))
print("Testing set has %d" %(X_test.shape[0]))
# Create global variables for cleaner code
f1_train = 'F1-score: training dataset'
acc_train = 'Accuracy: training dataset'
f1_test = 'F1-score: testing dataset'
acc_test = 'Accuracy: test dataset'

# Define train_predict and test_predict function to print out the result.
def train_predict(classifier, X_train, y_train):
    y_pred_train = classifier.predict(X_train)
    train_results = {}
    train_results[f1_train] = f1_score(y_train, y_pred_train)
    train_results[acc_train] = accuracy_score(y_train, y_pred_train)
    print (classifier.__class__.__name__)
    print(train_results)
    return train_results, y_pred_train

def test_predict(classifier, X_test, y_test):
    y_pred_test = classifier.predict(X_test)
    test_results = {}
    test_results[f1_test] = f1_score(y_test, y_pred_test) 
    test_results[acc_test] = accuracy_score(y_test, y_pred_test)
    print (classifier.__class__.__name__)
    print(test_results)
    return test_results, y_pred_test<jupyter_output><empty_output><jupyter_text>K- Neatest Neighbors **Now let's test out the KNN method**<jupyter_code># Find the k value from 1 to 21
error = []

for i in range(1, 21):
    classifier = KNeighborsClassifier(n_neighbors=i)
    KNN_classifier = classifier.fit(X_train, y_train)
    pred_i = classifier.predict(X_test)
    error.append(np.mean(pred_i != y_test))
# Plot a line chart to shows the trending for the k value
plt.figure(figsize=(8, 4))
plt.plot(range(1, 21), error, color='red', marker='o',markerfacecolor='blue')
plt.title('Error Rate K Value')
plt.xlabel('K Value')
plt.ylabel('Mean Error')
plt.show()
# Select the optimal k value and initiate KNeighborsClassifier
classifier_refined = KNeighborsClassifier(n_neighbors=3)

# Fit train data to the classifier
KNN_classifier = classifier_refined.fit(X_train, y_train)
# Print the result for KNeighborsClassifier
knn_output_train = train_predict(classifier, X_train, y_train)
knn_output_test = test_predict(classifier, X_test, y_test)<jupyter_output>KNeighborsClassifier
{'F1-score: training dataset': 0.7431832593532023, 'Accuracy: training dataset': 0.7563909774436091}
KNeighborsClassifier
{'F1-score: testing dataset': 0.7121573617517628, 'Accuracy: test dataset': 0.727883314119593}<jupyter_text>Logistic Regression **After getting the result from the benchmark(KNN method), let's check the logistic regression method. Let's start with the default setup first.**- for small dataset liblinear is a better choice<jupyter_code># Initiate LogisticRegression
lr_clf = LogisticRegression(solver='liblinear')

# Fit train data to the classifier
LR_classifier = lr_clf.fit(X_train, y_train)
# Print the result for LogisticRegression
lr_output_train = train_predict(LR_classifier, X_train, y_train)
lr_output_test = test_predict(LR_classifier, X_test, y_test)<jupyter_output>LogisticRegression
{'F1-score: training dataset': 0.8530961473389084, 'Accuracy: training dataset': 0.8694307196562836}
LogisticRegression
{'F1-score: testing dataset': 0.8516201117318436, 'Accuracy: test dataset': 0.86687384091023}<jupyter_text>**Let's test out some hyperparameters to check if we can get a better score*** random_state : The seed of the pseudo random number generator to use when shuffling the data (from default: 0 to 42)* penalty : Used to specify the norm used in the penalization (from default: l2 to l1)* C value: Inverse of regularization strength (from default: 1.0 to 100)<jupyter_code># Tuning for the hyperparameters
lr_clf = LogisticRegression(random_state=42, penalty = 'l1', C=100, solver='liblinear')
LR_classifier = lr_clf.fit(X_train, y_train)
lr_output_train = train_predict(LR_classifier, X_train, y_train)
lr_output_test = test_predict(LR_classifier, X_test, y_test)<jupyter_output>LogisticRegression
{'F1-score: training dataset': 0.8555505058961418, 'Accuracy: training dataset': 0.8702685284640171}
LogisticRegression
{'F1-score: testing dataset': 0.8559265811587793, 'Accuracy: test dataset': 0.8693799809533357}<jupyter_text>**From tuning the hyperparameters, we get a slightly better result for the logistic regression method. However, we would try another method to get an even better result.**  Random Forest **We will try the Random Forest method to see how far can we go**<jupyter_code># Initiate RandomForestClassifier
rf_clf = RandomForestClassifier()

# Fit train data to the classifier
RF_classifier= rf_clf.fit(X_train, y_train)
# Print the result for RandomForestClassifier
rf_output_train = train_predict(RF_classifier, X_train, y_train)
rf_output_test = test_predict(RF_classifier, X_test, y_test)<jupyter_output>RandomForestClassifier
{'F1-score: training dataset': 0.9935743522901375, 'Accuracy: training dataset': 0.9939634801288937}
RandomForestClassifier
{'F1-score: testing dataset': 0.8977350720658887, 'Accuracy: test dataset': 0.9029121347300887}<jupyter_text>**Let's test out some hyperparameters to check if we can get a better score*** random_state : Controls both the randomness of the bootstrapping of the samples used when building trees  (from default: None to 42)* n_estimators : The number of trees in the forest.(from default: 10 to 100)<jupyter_code># Tuning for the hyperparameters
rf_clf = RandomForestClassifier(random_state=42, n_estimators=100)
RF_classifier= rf_clf.fit(X_train, y_train)
rf_output_train = train_predict(RF_classifier, X_train, y_train)
rf_output_test = test_predict(RF_classifier, X_test, y_test)<jupyter_output>RandomForestClassifier
{'F1-score: training dataset': 0.9999771840562185, 'Accuracy: training dataset': 0.9999785177228786}
RandomForestClassifier
{'F1-score: testing dataset': 0.9077356130108424, 'Accuracy: test dataset': 0.9112826424740614}<jupyter_text>**After tuning the hyperparameters, we get a slightly better result for this method as well. I believe we are good to go.**  Result<jupyter_code># Print out all the result from these three methods
result = pd.DataFrame({'K - Nearest Neighbor': [knn_output_train[0][f1_train],knn_output_train[0][acc_train],knn_output_test[0][f1_test],knn_output_test[0][acc_test]],
                   'Logistic Regression': [lr_output_train[0][f1_train], lr_output_train[0][acc_train],lr_output_test[0][f1_test],lr_output_test[0][acc_test]],
                   'Random Forest': [rf_output_train[0][f1_train], rf_output_train[0][acc_train],rf_output_test[0][f1_test],rf_output_test[0][acc_test]]},
                  index=['F1-score_Train', 'Accuracy_Train','F1-score_Test', 'Accuracy_Test'])
print(result)<jupyter_output>K - Nearest Neighbor  Logistic Regression  Random Forest
F1-score_Train              0.743183             0.855551       0.999977
Accuracy_Train              0.756391             0.870269       0.999979
F1-score_Test               0.712157             0.855927       0.907736
Accuracy_Test               0.727883             0.869380       0.911283<jupyter_text>Based on the models above, we can see Random Forest got the highest f1-score and accuracy amount all the training and testing. So we will focus on its prediction.<jupyter_code># Plot a bar chart to check the feature importance
relative_importance = RF_classifier.feature_importances_
relative_importance = relative_importance / np.sum(relative_importance)

feature_importance =\
    pd.DataFrame(list(zip(features.columns,
                          relative_importance)),
                 columns=['feature', 'relativeimportance'])

feature_importance = feature_importance.sort_values('relativeimportance',
                                                    ascending=False)

feature_importance = feature_importance.reset_index(drop=True)

plt.figure(figsize=(7, 7))
sns.barplot(x='relativeimportance',
            y='feature',
            palette = sns.color_palette("Blues_d"),
            data=feature_importance)
plt.xlabel('Relative Importance')
plt.ylabel('Feature')
plt.title('Feature Importance (Random Forest)')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09820432545251719, 'fraction_numerical': 0.02828457303249487, 'mean_word_length': 5.369470547788219, 'pattern_counts': {'":': 0, '<': 82, '<?xml version=': 0, '>': 82, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '21', 'dir': 'jupyter-structured-clean-dedup', 'id': '645632', 'license': 'MIT', 'n_tokens_mistral': 8928, 'n_tokens_neox': 8148, 'n_words': 2906, 'path': 'Starbucks_Capstone_notebook.ipynb', 'repo_name': 'arrickx/StarbuckProject', 'size': '27789'}