<jupyter_start><jupyter_text>Ad recommender:In this notebook, I have implemented a logistic regression algorithm(pretty much from the scratch) for predicting whether a person will buy a magazine or not, based on their gender, age, salary, and earlier purchases.<jupyter_code># Imports:
import numpy as np
import pandas as pd
# For interactive plots, change 'inline' to 'notebook'.
%matplotlib inline
from matplotlib import pyplot as plt<jupyter_output><empty_output><jupyter_text>Loading the dataset:Dataset obtained from Kaggle.<jupyter_code>df = pd.read_csv('data.csv')
display(df)<jupyter_output><empty_output><jupyter_text>Data preprocessing:<jupyter_code># 'User ID' is a useless feature.
df = df.drop(columns = ['User ID'])
df.isnull().sum()<jupyter_output><empty_output><jupyter_text>No null values are present.<jupyter_code>df["Purchased"].value_counts()<jupyter_output><empty_output><jupyter_text>A value of 1 means that they have purchased the magazine.<jupyter_code># We need numeric data.
df["Gender"].replace('Male', 0, inplace=True) # 0 is for male
df["Gender"].replace('Female', 1, inplace=True) # 1 is for female<jupyter_output><empty_output><jupyter_text>Finally, we have 3 features.<jupyter_code>X = df[["Age", "EstimatedSalary"]] 
y = df[["Purchased"]] # Dependent variable.

# Feature scaling: Standard scaling/ Mean normalization
X = (X - X.mean())/X.std()

X = X.assign(x_0 = pd.Series(np.ones(X.shape[0])).values) # Adding the dummy column filled with ones.
X = X.assign(Gender = df[["Gender"]]) # Adding the gender column.
display(X)<jupyter_output><empty_output><jupyter_text>We have added the dummy feature column filled with ones. (To take care of $\theta _0$)  Splitting the dataset:We split the dataset into training and validation datasets.<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42) # 3:1 Ratio split.

m = X_train.shape[0] # The number of datapoints/rows.
n = X_train.shape[1] # The number of columns.<jupyter_output><empty_output><jupyter_text>Training:<jupyter_code>theta = np.zeros(n) # Initial value for the parameters.
#theta_temp = theta # Temporary theta required for simultaneous upgrading.<jupyter_output><empty_output><jupyter_text>Helper functions:<jupyter_code># The logistic function/ sigmoid function:
def sigmoid(z):
    return np.float32(1/(1 + np.exp(-z)))
# The cost function:
def cost(y, X, theta):
    s = 0
    for i in range(m):
        s = s - (1/m)*(y[i]*np.log(h(X[i], theta)) + (1 - y[i])*np.log(1 - h(X[i], theta)))
    return s
# Our hypothesis function:
def h(x, theta): # Here x is a datapoint.
    return sigmoid(np.dot(x,theta))
# Calculates the gradient with respect to a parameter.
def gradient(X, y, j):
    s = 0
    for i in range(m):
        s = s + (1/m)*(h(X[i], theta) - y[i])*X[i][j]
    return s<jupyter_output><empty_output><jupyter_text>Gradient Descent:<jupyter_code># Converting dataframes to numpy arrays:
X_train = X_train.values 
y_train = y_train.values
X_test = X_test.values 
y_test = y_test.values
y_test = y_test.reshape(y_test.shape[0],)
# GRADIENT DESCENT PARAMETERS:
N_ITER = 100 # The number of iterations.
alpha = 0.3 # The learning rate.
# GRADIENT DESCENT:
costY = []
for i in range(N_ITER):
    costY.append(cost(y_train, X_train, theta))
    # Theta gets updated simultaneously in each iteration and then gets updated as a whole.
    theta = [(theta[j] - alpha*gradient(X_train, y_train, j))[0] for j in range(n)]
print("The parameters are: ", theta)
# Plotting the cost function for better visualization: 
plt.plot(np.linspace(1, N_ITER, N_ITER), costY)
plt.show()
predictions1 = [h(X_test[i], theta) for i in range(X_test.shape[0])]
# Cut-off set as 0.5
predictions = np.round(predictions1)
# Gives us some measure of the accuracy of our model.
def score(a, b):
    return ((np.sum(a==b))/y_test.shape[0])
print("The accuracy is: {:.2f}".format(score(predictions, y_test)))
from sklearn import metrics
cnf_matrix = metrics.confusion_matrix(y_test, predictions)
print(cnf_matrix)
print("Accuracy:",metrics.accuracy_score(y_test, predictions))
# Another way to test the accuracy of our classifier:
auc = metrics.roc_auc_score(y_test, predictions1) # Note that this accepts probability predictions and not binary integer classes.
print(auc)<jupyter_output>0.9631059631059632
{'fraction_non_alphanumeric': 0.1270819073693817, 'fraction_numerical': 0.013917408167921514, 'mean_word_length': 5.622356495468278, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '261207', 'license': 'MIT', 'n_tokens_mistral': 1593, 'n_tokens_neox': 1516, 'n_words': 431, 'path': 'Logistic-Regression/Recommender.ipynb', 'repo_name': 'hrshtv/ML-Algorithms', 'size': '4383'}