<jupyter_start><jupyter_text>Working with Social Network Classification Problem :: (LR, kNN, DT, SVM)*<NAME> S**** **Problem statement:**Given is a dataset of a social network sales company, which contains user id, gender, age, salary of a customer. Whether or not the customer purchased from the site is identified with "Purchased" column having values {0, 1}, where 0 represents "not purchased" and 1 represents "Purchased.**Purpose::**Design models using Logistic Regression, kNN algorithms, SVM, Decision Trees to predict whether a customer will purchase or not.This is an example of Binary classification problem.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
data = pd.read_csv('./datasets/Social_Network_Ads.csv')
data.head()<jupyter_output><empty_output><jupyter_text>Understanding Data***<jupyter_code>data.info()
data.shape
data.isna().sum()<jupyter_output><empty_output><jupyter_text>No Null values present in dataset<jupyter_code>#Checking gender
data.Gender.value_counts(normalize=True)<jupyter_output><empty_output><jupyter_text>In this dataset female customers consitute 51% while male customers constitute 49%. Since the 'Gender' column is an 'Object' type, it needs to be encoded using OneHot Encoding.<jupyter_code>data.Purchased.value_counts(normalize=True)<jupyter_output><empty_output><jupyter_text>64% in this dataset purchased items from website and 36% not purchased **OneHot Encoding*****<jupyter_code>data = pd.get_dummies(data = data, columns = ['Gender'])
data.head()<jupyter_output><empty_output><jupyter_text>**Splitting data*****<jupyter_code>from sklearn.model_selection import train_test_split
x = data.drop(columns=['User ID', 'Purchased'], axis = 1)
y = data['Purchased']
x_train, x_test, y_train, y_test = train_test_split(x, y, random_state=42, test_size=0.2)
print(x_train.shape)
print(y_train.shape)
import warnings
warnings.filterwarnings("ignore")<jupyter_output><empty_output><jupyter_text>1. Logistic Regression Classifier***<jupyter_code>from sklearn.linear_model import LogisticRegression
from sklearn.metrics import f1_score, accuracy_score, precision_score, recall_score, confusion_matrix
def generate_logi_model(x_train, y_train, x_test):
    logi_model = LogisticRegression()
    logi_model.fit(x_train, y_train)
    return logi_model.predict(x_test)
def check_model_metrices(y_test, y_pred):
    print('Model Accuracy = ', accuracy_score(y_test, y_pred))
    print('Model Precision = ', precision_score(y_test, y_pred))
    print('Model Recall = ', recall_score(y_test, y_pred))
    print('Model F1 Score = ', f1_score(y_test, y_pred))
    print('Confusion Matrix = \n', confusion_matrix(y_test, y_pred))
#Try 1
lr_y_pred = generate_logi_model(x_train, y_train, x_test)
check_model_metrices(y_test, lr_y_pred)<jupyter_output>Model Accuracy =  0.65
Model Precision =  0.0
Model Recall =  0.0
Model F1 Score =  0.0
Confusion Matrix = 
 [[52  0]
 [28  0]]<jupyter_text>2. kNN Classifier***<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
def generate_kNN_model(x_train, y_train, x_test, k):
    knn_model = KNeighborsClassifier(n_neighbors=k, metric='minkowski')
    knn_model.fit(x_train, y_train)
    return knn_model.predict(x_test)
# Optimizing 'k' or 'n-neighbers' value
def find_optimal_k(x_train, y_train, x_test):
    accur_dict = dict()
    for k in np.arange(3,16):
        y_pred = generate_kNN_model(x_train, y_train, x_test, k)
        accur_dict[k] = accuracy_score(y_test, y_pred)
    #Plot
    print('Various k values accuracies :: ', accur_dict)
    plt.plot(list(accur_dict.keys()),list(accur_dict.values()), marker ='o')
    plt.title('k-Values vs Accuracy')
    plt.show()
    optimal_k = max(accur_dict, key = lambda x: accur_dict[x])
    print('Best k value = ', optimal_k)
    return optimal_k
# Find Optimal k value
k_optimal = find_optimal_k(x_train, y_train, x_test)
# Model with k = 5 and check metrices
knn_y_pred = generate_kNN_model(x_train, y_train, x_test, k_optimal)
print(f'kNN classifier with k = {k_optimal} has :: \n')
check_model_metrices(y_test, knn_y_pred)<jupyter_output>kNN classifier with k = 5 has :: 

Model Accuracy =  0.825
Model Precision =  0.7692307692307693
Model Recall =  0.7142857142857143
Model F1 Score =  0.7407407407407408
Confusion Matrix = 
 [[46  6]
 [ 8 20]]<jupyter_text>kNN classifer has produced an accuracy of 82.5%  Scale and Re-Test to improve accuracy***<jupyter_code>from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
x_train = scaler.fit_transform(x_train)
x_test = scaler.fit_transform(x_test)
# Recheck Logistic Regression
lr_y_pred = generate_logi_model(x_train, y_train, x_test)
check_model_metrices(y_test, lr_y_pred)<jupyter_output>Model Accuracy =  0.875
Model Precision =  0.875
Model Recall =  0.75
Model F1 Score =  0.8076923076923077
Confusion Matrix = 
 [[49  3]
 [ 7 21]]<jupyter_text>Accuracy of the model has gone to 87.5%<jupyter_code># Recheck kNN classifier
k_optimal = find_optimal_k(x_train, y_train, x_test)
knn_y_pred = generate_kNN_model(x_train, y_train, x_test, k_optimal)
check_model_metrices(y_test, knn_y_pred)<jupyter_output>Various k values accuracies ::  {3: 0.925, 4: 0.925, 5: 0.925, 6: 0.925, 7: 0.925, 8: 0.925, 9: 0.925, 10: 0.925, 11: 0.925, 12: 0.925, 13: 0.925, 14: 0.925, 15: 0.925}<jupyter_text>Accuracy of kNN model has increased to 92.5%  3. Decision Tree Classifier***<jupyter_code>from sklearn.tree import DecisionTreeClassifier
dt_model = DecisionTreeClassifier()
dt_model.fit(x_train, y_train)
dt_pred = dt_model.predict(x_test)
check_model_metrices(y_test, dt_pred)<jupyter_output>Model Accuracy =  0.8125
Model Precision =  0.7096774193548387
Model Recall =  0.7857142857142857
Model F1 Score =  0.7457627118644068
Confusion Matrix = 
 [[43  9]
 [ 6 22]]<jupyter_text>4.  SVM Classifier***<jupyter_code>from sklearn.svm import SVC
# Linear Model
svc_linear = SVC(kernel='linear')
svc_linear.fit(x_train, y_train)
svc_linear_pred = svc_linear.predict(x_test)
check_model_metrices(y_test, svc_linear_pred)
# Polynomial Model
svc_poly = SVC(kernel='poly', degree = 4)
svc_poly.fit(x_train, y_train)
svc_poly_pred = svc_poly.predict(x_test)
check_model_metrices(y_test, svc_poly_pred)
# RBF Model
svc_rbf = SVC(kernel='rbf')
svc_rbf.fit(x_train, y_train)
svc_rbf_pred = svc_rbf.predict(x_test)
check_model_metrices(y_test, svc_rbf_pred)<jupyter_output>Model Accuracy =  0.9125
Model Precision =  0.8387096774193549
Model Recall =  0.9285714285714286
Model F1 Score =  0.8813559322033899
Confusion Matrix = 
 [[47  5]
 [ 2 26]]
{'fraction_non_alphanumeric': 0.10872544540886249, 'fraction_numerical': 0.05329678696512868, 'mean_word_length': 6.289678135405105, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '232228', 'license': 'MIT', 'n_tokens_mistral': 2692, 'n_tokens_neox': 2423, 'n_words': 565, 'path': 'notebooks/ex-social-network-classification-problem.ipynb', 'repo_name': 'sreeh4ri/ictk_projects', 'size': '6571'}