<jupyter_start><jupyter_text>Credit Card Fraud Detection Project The datasets contains transactions made by credit cards in September 2013 by european cardholders.This dataset presents transactions that occurred in two days, where we have 492 frauds out of 284,807 transactions. The dataset is highly unbalanced, the positive class (frauds) account for 0.172% of all transactions. The dataset contains only numerical input variables which are the result of a PCA transformation. Unfortunately, due to confidentiality issues, the owners of the dataset cannot provide the original features and more background information about the data. Features V1, V2, … V28 are the principal components obtained with PCA, the only features which have not been transformed with PCA are 'Time' and 'Amount'. Feature **'Time'** contains the seconds elapsed between each transaction and the first transaction in the dataset. The feature **'Amount'** is the transaction Amount, this feature can be used for example-dependant cost-senstive learning. Feature 'Class' is the response variable and it takes value 1 in case of fraud and 0 otherwise. - Link to the dataset: https://www.kaggle.com/mlg-ulb/creditcardfraud<jupyter_code># Importing libraries
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
%matplotlib inline
from sklearn import metrics
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import plot_confusion_matrix, classification_report, plot_roc_curve
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.utils import class_weight
from imblearn.under_sampling import RandomUnderSampler
import shap
# Setting params for plots
from matplotlib import cycler
colors = cycler('color',
                ['#EE6666', '#3388BB', '#9988DD',
                 '#EECC55', '#88BB44', '#FFBBBB'])
plt.rc('axes', facecolor='#E6E6E6', edgecolor='none',
       axisbelow=True, grid=True, prop_cycle=colors)
plt.rc('grid', color='w', linestyle='solid')
plt.rc('xtick', direction='out', color='gray')
plt.rc('ytick', direction='out', color='gray')
plt.rc('patch', edgecolor='#E6E6E6')
plt.rc('lines', linewidth=2)<jupyter_output><empty_output><jupyter_text>Obtaining<jupyter_code># Loading in dataset into a dataframe
df = pd.read_csv('../../../data/creditcard.csv')
df.head()<jupyter_output><empty_output><jupyter_text>Exploring<jupyter_code># Viewing the number of columns and rows within the dataset
print(f"Credit Card Fraud Detection Dataset:\nrows: {df.shape[0]}\ncolumns: {df.shape[1]}")
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 284807 entries, 0 to 284806
Data columns (total 31 columns):
Time      284807 non-null float64
V1        284807 non-null float64
V2        284807 non-null float64
V3        284807 non-null float64
V4        284807 non-null float64
V5        284807 non-null float64
V6        284807 non-null float64
V7        284807 non-null float64
V8        284807 non-null float64
V9        284807 non-null float64
V10       284807 non-null float64
V11       284807 non-null float64
V12       284807 non-null float64
V13       284807 non-null float64
V14       284807 non-null float64
V15       284807 non-null float64
V16       284807 non-null float64
V17       284807 non-null float64
V18       284807 non-null float64
V19       284807 non-null float64
V20       284807 non-null float64
V21       284807 non-null float64
V22       284807 non-null float64
V23       284807 non-null float64
V24       284807 non-null float64
V25       284807 non-null float64
V26  [...]<jupyter_text>At first glance, it seems we do not have any rows with null information.<jupyter_code>df.describe()
# We have rows where Amount is equal to 0. Let's drop these from the df
print(f"Number of rows where Amount is $0.00: \n{len(df[df['Amount'] == 0])}")
# Dropping rows where Amount is equal to 0
df = df[df['Amount'] > 0]
fraud = df['Class'].value_counts()[1]
true = df['Class'].value_counts()[0]
print(f"Percentage of fraudulent transactions: {round(fraud/len(df) * 100, 3)}%\n"
      f"Percentage of non-fraudulent transactions: {round(true/len(df) * 100, 3)}%")
ax = sns.countplot(x='Class', data=df)
ax.set_title('Viewing Class Imbalance');<jupyter_output><empty_output><jupyter_text>There is extreme class imbalance between the two classes. Remember that the number of fraudulent transactions within this dataset is 492, which accounts for only 0.173% of all the data.<jupyter_code>ax = sns.histplot(data=df, x='Time')
ax.set_title('Distribution of Time Between All Transactions & First Transaction');
df['Time'].describe()
n_sec_per_day = 86400

df['Time'].max() / n_sec_per_day<jupyter_output><empty_output><jupyter_text>We see that the time column contains around 285,000 transactions by looking at the 'count' row. By taking the maximum value of the time column and dividing it by the number of seconds in a day, we can see that all transactions within this dataset occurred within 2 days.  Creating an 'Hours' column We are going to create a new column called **'Hours'** that tells us what hour the transaction happened in military time.<jupyter_code>def make_hours(num):
    val = round((num / n_sec_per_day * 24)) 
    if val > 24:
        val = val - 24
    return val
apply_hours = df['Time'].apply(make_hours)
df['Hours'] = apply_hours
plt.figure(figsize=(10,5))
sns.countplot(x='Hours', data=df);
df['Hours'].value_counts()
# The amount column is not a normal distribution
plt.figure(figsize=(15,5))
sns.kdeplot(df.loc[df['Class']==0, 'Amount'], label='Legitimate')
sns.kdeplot(df.loc[df['Class']==1, 'Amount'], label='Fraud')
plt.legend();<jupyter_output><empty_output><jupyter_text>We notice there are some major outliers in the 'Amount' column. It seems like the majority of the fraudulent charges are below $3000.00 charges.<jupyter_code>df['Amount'].describe()
high = df['Amount'].quantile(0.99)

print(f"Number of rows containing 'Amount' greater than the 99th percentile:\n"
      f"{len(df[df['Amount'] > high])}")
# Removing rows from the dataframe that have 'Amount' that are greater than 
# the 99th percentile
df = df[df['Amount'] <= high]
# Dropping unecessary columns
df.drop(columns=['Time'], inplace=True)
df_importance = pd.DataFrame(df.corr()['Class']).\
                sort_values(by='Class',ascending=False)
df_importance.plot(kind='barh', figsize=(15,5))
plt.title('Significant Features When Determining Fraudulent and Non-fraudulent Charges');<jupyter_output><empty_output><jupyter_text>Undersampling the data In order to make sure our model does not guess the majority class when training on the data, we will undersample the data to better balance the classes. We should note that undersampling has drawbacks, one of which is that we lose an enormous amount of data.<jupyter_code>X = df.drop(columns='Class')
Y = df['Class']
# Undersampling the data
undersample = RandomUnderSampler(sampling_strategy=0.5)
feats, labels = undersample.fit_resample(X=X, y=Y)<jupyter_output><empty_output><jupyter_text>Creating train and test sets We will now create train and test sets on the undersampled data.<jupyter_code># Creating a train and test set
X_train, X_test, y_train, y_test = train_test_split(feats, labels, random_state=42)
X_train.columns
y_train.value_counts().plot(kind='bar');<jupyter_output><empty_output><jupyter_text>We see that the classes are now not as extremely unbalanced before we implemented undersampling.  Dummy Classifier<jupyter_code>dummy = DummyClassifier(strategy='stratified')
dummy.fit(X_train, y_train)
y_pred = dummy.predict(y_test)
# Classification report
print(f"Recall Score: {metrics.recall_score(y_test, y_pred)}")
print(metrics.classification_report(y_test, y_pred))

plot_confusion_matrix(dummy, X_test,y_test,normalize='true',cmap='Blues');<jupyter_output>Recall Score: 0.44144144144144143
              precision    recall  f1-score   support

           0       0.72      0.69      0.71       231
           1       0.41      0.44      0.42       111

    accuracy                           0.61       342
   macro avg       0.56      0.57      0.57       342
weighted avg       0.62      0.61      0.61       342<jupyter_text>The recall score for the dummy classifier is around 40% when it comes to correctly identifying fraudulent charges. The dummy model tends to be guessing more towards the non-fraudulent class. Let's see if we can improve the recall score through Logistic Regression.  Adding Class Weights Because our two classes are still unabalanced even after undersampling the data, we will be using class weights to help the model better differentiate between the classes.<jupyter_code>class_weights = class_weight.compute_class_weight('balanced', classes=np.unique(y_train), 
                                                  y=y_train)
class_weights_dict = {0: class_weights[0], 1: class_weights[1]}
print(f"Our class weights:\n{class_weights_dict}")<jupyter_output>Our class weights:
{0: 0.7533039647577092, 1: 1.4869565217391305}<jupyter_text>Logistic Regression The first type of model we will use to train on the data is the Logistic Regression model. This model excels at fraud detection, and is rather easy to interpret when compared to other models like RandomForest or XGBoost Classifier models.<jupyter_code># Creating and fitting the model on the train set
model = LogisticRegression(fit_intercept=False, C=1, solver='liblinear', 
                          class_weight=class_weights_dict, penalty='l2')
def evaluate_model(model, X_train, y_train, X_test, y_test):
    """
    Definition:
    fits model on the training set and evaluates the model on the testing set.
    Prints out classification report along with a confusion matrix.
    
    Args:
    model: Required. A model to evaluate.
    X_train: Required. Features of the training set
    y_train: Required. Labels of the training set
    X_test: Required. Features of the testing set
    y_test: Required. Labels of the testing set
    
    Returns:
    A Classification report and confustion matrix of model evaluation on the test
    set.    
    """
    
    fit_model = model.fit(X_train, y_train)
    # creating y_pred values for train and test sets
    y_hat_test = fit_model.predict(X_test)
    y_hat_train = fit_model.predict(X_train)
    # comparing accuracy and recall score s of the train and test sets
    print(f"Train accuracy score: {metrics.balanced_accuracy_score(y_train, y_hat_train)}")
    print(f"Test accuracy score: {metrics.balanced_accuracy_score(y_test, y_hat_test)}")
    print(f"\nTrain recall score: {metrics.recall_score(y_train, y_hat_train)}")
    print(f"Test recall score: {metrics.recall_score(y_test, y_hat_test)}")
    
    # printing out classification report and plotting confusion matrix
    print("Classification Report")
    print('---' * 8)
    classification_report(y_test, y_hat_test)
    plot_confusion_matrix(fit_model, X_test,y_test,normalize='true',cmap='Reds');
    
    ## Plot Roc Curve on second ax
    plt.figure(figsize=(7, 7))
    roc_plot = plot_roc_curve(fit_model, X_test, y_test)
    plt.legend()
    # Threshold line
    plt.plot([0,1],[0,1],ls=':')
    plt.title('ROC Curve')
    plt.show()
evaluate_model(model, X_train, y_train, X_test, y_test)<jupyter_output>Train accuracy score: 0.9381472259464981
Test accuracy score: 0.9468234468234469

Train recall score: 0.9159420289855073
Test recall score: 0.9369369369369369
Classification Report
------------------------<jupyter_text>We see that the logistic regression model performed well on the test set, with a recall score of 94% on the fraudulent class.  Decision Tree Classifier<jupyter_code>tree = DecisionTreeClassifier()
evaluate_model(tree, X_train, y_train, X_test, y_test)<jupyter_output>Train accuracy score: 1.0
Test accuracy score: 0.9241254241254241

Train recall score: 1.0
Test recall score: 0.8828828828828829
Classification Report
------------------------<jupyter_text>Random Forest Classifier<jupyter_code>forest = RandomForestClassifier()
evaluate_model(forest, X_train, y_train, X_test, y_test)<jupyter_output>Train accuracy score: 1.0
Test accuracy score: 0.9304434304434304

Train recall score: 1.0
Test recall score: 0.8738738738738738
Classification Report
------------------------
{'fraction_non_alphanumeric': 0.09527249377959708, 'fraction_numerical': 0.05281322738582551, 'mean_word_length': 4.4032957502168255, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 55, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '521398', 'license': 'MIT', 'n_tokens_mistral': 4270, 'n_tokens_neox': 3792, 'n_words': 1270, 'path': 'creditcard_fraud.ipynb', 'repo_name': 'Lewis34cs/Credit_Card_Fraud_Detection', 'size': '12459'}