<jupyter_start><jupyter_text>Identifying Fraudulent Activity

Company XYZ is an e-commerce site that sells hand-made clothes.

You have to build a model that predicts whether a user has a high probability of using the site to perform some illegal activity or not. This is a super common task for data scientists.

You only have information about the user first transaction on the site and based on that you have to make your classification (“fraud/no fraud”).

These are the tasks you are asked to do:
1. For each user, determine their country based on the IP address
2. Build a model to predict whether an activity is fraudulent or not. Explain how different assumptions about the cost of false positives vs false negatives would impact the model
3. Your boss is a bit worried about using a model she doesn’t understand for something as important as fraud detection. How would you explain her how the model is making the predictions? Not from a mathematical perspective (she couldn’t care less about that), but from a user perspective. What kinds of users are more likely to be classified as at risk? What are their characteristics?
4. Let’s say you now have this model which can be used live to predict in real time if an activity is fraudulent or not. From a product perspective, how would you use it? That is, what kind of different user experiences would you build based on the model output?  Data Description

- **user_id:** Id of the user. Unique by user
- **signup_time:** the time when the user created her account (GMT time)
- **purchase_time:** the time when the user bought the item (GMT time)
- **purchase_value:** the cost of the item purchased (USD)
- **device_id:** the device id. You can assume that it is unique by device. I.e., same device ID means that the same physical device was used for the transaction
- **source:** user marketing channel: ads, SEO, Direct (i.e. came to the site by directly typing the site address on the browser)
- **browser:** the browser used by the user
- **sex:** user sex: Male/Female
- **age:** user age
- **ip_address:** user numeric ip address
- **class:** this is what we are trying to predict: whether the activity was fraudulent (1) or not (0)  Navigation
1. [Initial Exploration](Initial)
2. Case Study Questions
    1. [Determine User Country](Country)
    2. [Model to Predict Fraud](Model)
        - [Feature Engineering](Engineering)
        - [Modeling](Modeling)
    3. [Model Explanation](Explanation)
    4. [Using the Model in Production](Production)
3. [Conclusion](Conclusion) 
 Initial Exploration<jupyter_code>!pip install category_encoders
!pip install pdpbox
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import train_test_split
import category_encoders as ce
from sklearn.metrics import roc_curve
import matplotlib.pyplot as plt
from pdpbox import pdp
from pdpbox import info_plots
df =  pd.read_csv('PRIVATE CSV')

df['ip_address'] = df['ip_address'].astype(int)
df['signup_time'] = pd.to_datetime(df['signup_time'])
df['purchase_time'] = pd.to_datetime(df['purchase_time'])

print(df.shape)
df.head()
df.info()
ip_bounds_and_country = pd.read_csv('PRIVATE CSV')

print(ip_bounds_and_country.shape)
ip_bounds_and_country.head()
ip_bounds_and_country['lower_bound_ip_address'] = ip_bounds_and_country['lower_bound_ip_address'].astype(int)<jupyter_output><empty_output><jupyter_text>Identify Each User's Country<jupyter_code>def find_country_by_ip_address(ip_address):
    country = list(ip_bounds_and_country[
            (ip_bounds_and_country['lower_bound_ip_address'] <= ip_address) & 
            (ip_address <= ip_bounds_and_country['upper_bound_ip_address'])
            ]['country'])
    if len(country) == 1:
        return country[0]
    return 'Not Found'

df['country'] = df['ip_address'].apply(find_country_by_ip_address)
df.head()
df['country'].value_counts(dropna=False)<jupyter_output><empty_output><jupyter_text>Build a Model to Predict Fraud 
 Feature Engineering<jupyter_code>df['time_to_purchase'] = (df['purchase_time'] - df['signup_time']).astype(int)
def unique_ip(ip_address):
    return ip_counts[ip_address]

ip_counts = df['ip_address'].value_counts()
df['unique_ip'] = df['ip_address'].apply(unique_ip)
def unique_device(device_id):
    return device_counts[device_id]

device_counts = df['device_id'].value_counts()
df['unique_device'] = df['device_id'].apply(unique_device)
def day_of_week(date):
    return date.dayofweek

def week_of_year(date):
    return date.week

df['signup_day'] = df['signup_time'].apply(day_of_week)
df['signup_week'] = df['signup_time'].apply(week_of_year)
df['purchase_day'] = df['purchase_time'].apply(day_of_week)
df['purchase_week'] = df['purchase_time'].apply(week_of_year)
df.head()<jupyter_output><empty_output><jupyter_text>Model Building<jupyter_code>bottom_countries = df.groupby('country').size().sort_values(ascending=False)[50:].index
x = dict.fromkeys(bottom_countries, 'Other')
df['country'] = df['country'].replace(x)

X, y = df.drop(['class', 'signup_time',
                'purchase_time', 'device_id',], axis='columns'), df['class']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33,
                                                    random_state=42)<jupyter_output><empty_output><jupyter_text>Get a baseline accuracy by looking at the majority classifier. 90.6% is our baseline. Not surprisingly most people don't commit fraud, but 9.4%% fraud rate does seem high.<jupyter_code>y_train.value_counts(normalize=True, dropna=False)
def one_hot_encode(df, cols):
    df = df.copy()
    cols.insert(0, 'user_id')
    one_hot_encoder = ce.OneHotEncoder(use_cat_names=True)
    one_hots = one_hot_encoder.fit_transform(df[cols])
    df = df.drop(cols[1:], axis='columns')
    df = df.merge(one_hots, on='user_id')
    return df

X_train = one_hot_encode(X_train, ['browser', 'source', 'sex', 'country'])
X_test = one_hot_encode(X_test, ['browser', 'source', 'sex', 'country'])
rf_model = RandomForestClassifier(n_estimators=100, max_features=3,
                                  oob_score=True)
rf_model.fit(X_train, y_train)
print('OOB accuracy:', rf_model.oob_score_)
print('OOB Confusion Matrix\n')
print(pd.DataFrame(confusion_matrix(y_train, 
                                  rf_model.oob_decision_function_[:,1].round(),
                                  labels=[0, 1])))
print('Test accuracy:', rf_model.score(X_test, y_test))
print('Test Set Confusion Matrix\n')
print(pd.DataFrame(confusion_matrix(y_test, rf_model.predict(X_test),
                                    labels=[0, 1])))
pred_prob = rf_model.predict_proba(X_test)[:,1]
fpr, tpr, thresholds = roc_curve(y_test, pred_prob)

plt.plot(fpr, tpr)
plt.plot([0, 1], [0, 1])
plt.ylim([0, 1.0])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('ROC')
plt.show()
error_cutoff = pd.DataFrame({'cutoff': pd.Series(thresholds),
                             'class0_error(fpr)': pd.Series(fpr),
                             'class1_error(fnr)': 1 - pd.Series(tpr)})
error_cutoff['optimal_value'] = 1 - error_cutoff['class1_error(fnr)'] - error_cutoff['class0_error(fpr)']
print(error_cutoff.sort_values('optimal_value', ascending=False).head(1))
pdp_dict = {'Country': [],
            'Age': 'age',
            'Purchase Value': 'purchase_value',
            'Time to Purchase': 'time_to_purchase',
            'Unique IP': 'unique_ip',
            'Unique Device': 'unique_device',
            'Signup Day': 'signup_day',
            'Signup Week': 'signup_week',
            'Purchase Day': 'purchase_day',
            'Purchase Week': 'purchase_week',
            'Browser': [],
            'Source': [],
            'Sex': []}

for col in X_train.columns:
    if 'country' in col:
        pdp_dict['Country'].append(col)
    elif 'browser' in col:
        pdp_dict['Browser'].append(col)
    elif 'source' in col:
        pdp_dict['Source'].append(col)
    elif 'sex' in col:
        pdp_dict['Sex'].append(col)
# singles = ['Age', 'Purchase Value', 'Time to Purchase', 'Unique IP', 
#            'Unique Device', ' Signup Day', 'Signup Week', 
#            'Purchase Day', 'Purchase Week']

# for key, value in pdp_dict.items():
#     pdp_iso = pdp.pdp_isolate(
#                 model=rf_model, 
#                 dataset=X_train,      
#                 model_features=list(X_train), 
#                 feature=pdp_dict[key], 
#                 num_grid_points=50)
#     if key in singles:
#         pdp_dataset = pd.Series(pdp_iso.pdp, index=pdp_iso.feature_grids)
#         pdp_dataset.plot(title=key)
#     else:
#         pdp_dataset = pd.Series(pdp_iso.pdp, index=pdp_iso.display_columns)
#         pdp_dataset.sort_values(ascending=False).plot(kind='bar', title=key)
#     plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11873819837831834, 'fraction_numerical': 0.006109074752860158, 'mean_word_length': 3.753959873284055, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '434753', 'license': 'MIT', 'n_tokens_mistral': 2923, 'n_tokens_neox': 2856, 'n_words': 772, 'path': '04. Identifying_Fraudulent_Activity.ipynb', 'repo_name': 'bundickm/Data-Masked-Case-Studies', 'size': '9003'}