<jupyter_start><jupyter_text>Breast Cancer Classifier from scratch  Load the required libraries<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import accuracy_score<jupyter_output><empty_output><jupyter_text>Load and Read the dataset Change the columns name also<jupyter_code>df = pd.read_csv("datasets/breast-cancer-wisconsin.data", header = None)
df.rename(columns = {0:"id", 
                     1:"clump-thickness", 
                     2:"cell-size", 
                     3:"cell-shape", 
                     4:"marginal-adhesion", 
                     5:"epithelial-cell-size", 
                     6:"bare-nuclei", 
                     7:"bland-chromatin", 
                     8:"normal-nucleoli", 
                     9:"mitoses", 
                     10:"class"}, 
          inplace = True)
df.head(10)<jupyter_output><empty_output><jupyter_text>Count the observations for different classes<jupyter_code>df['class'].value_counts()
# 2 is for benign cancer
# 4 is for malignant cancer<jupyter_output><empty_output><jupyter_text>Create input matrix and labels for the given dataset<jupyter_code>label_vector = df.iloc[:, 10]   #class labels: 2 = benign, 4 = malignant
feature_vector = df.iloc[:, 1:10] #features vectors
feature_vector<jupyter_output><empty_output><jupyter_text>Relabel the observed values as 0 and 1<jupyter_code># type(label_vector)
label_vector = label_vector.replace(2,0)
label_vector = label_vector.replace(4,1)
label_vector.value_counts()<jupyter_output><empty_output><jupyter_text>Data Preprocessing<jupyter_code>print("Does the feature set contain any null values:",feature_vector.isnull().values.any())
print("\nFeature set information")
feature_vector.info()<jupyter_output>Does the feature set contain any null values: False

Feature set information
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 699 entries, 0 to 698
Data columns (total 9 columns):
 #   Column                Non-Null Count  Dtype 
---  ------                --------------  ----- 
 0   clump-thickness       699 non-null    int64 
 1   cell-size             699 non-null    int64 
 2   cell-shape            699 non-null    int64 
 3   marginal-adhesion     699 non-null    int64 
 4   epithelial-cell-size  699 non-null    int64 
 5   bare-nuclei           699 non-null    object
 6   bland-chromatin       699 non-null    int64 
 7   normal-nucleoli       699 non-null    int64 
 8   mitoses               699 non-null    int64 
dtypes: int64(8), object(1)
memory usage: 49.3+ KB<jupyter_text>The last cell says that the values for the feature "bare-nuclei" is an object, which is not compatible for our calculations.  So we need to change it into the int64 type, so that it becomes homogeneous with the entire dataset.<jupyter_code>feature_vector.mean()<jupyter_output><empty_output><jupyter_text>The last cell does not contain the mean value for the "bare-nuclei" feature, as it is of type object.  Lets try to change the type for this.<jupyter_code># feature_vector["bare-nuclei"].astype('int64')<jupyter_output><empty_output><jupyter_text>We got an error while trying to change the type for the "bare-nuclei" feature.  ValueError: invalid literal for int() with base 10: '?'  This means that there are some entries within that feature which contains '?' as values (hence incompatible).<jupyter_code># tmp = feature_vector["bare-nuclei"].where(lambda x : x != '?')
# tmp.str.contains("?", regex=False).value_counts()
feature_vector["bare-nuclei"].str.contains("?", regex=False).value_counts()<jupyter_output><empty_output><jupyter_text>There are 16 such entries which are making the "bare-nuclei" feature incompatible for modelling.  Now we have to get rid of such entries first before changing the type for that feature.  One possible way of dealing with such entries would be to drop them and all the corresponding entries of the other features as well.  But doing so would leave us with lesses data for modelling.  We will even lose the patterns or insights that these entries might be holding to.  There is another way to solve this issue, where we can replace the incompatible entries with the mean of all the entries for that particular feature.  This will save us from lossing the insights into the dataset.  Although dropping the entries would be advisable if the number of incompatible entries are insignificant than the voulume of the entire dataset.<jupyter_code>def get_mean_for_bare_nulcei():
    tmp = feature_vector["bare-nuclei"].to_numpy()
    sum = 0
    for n in tmp:
        if (n == '?'):
            continue
        else:
            sum += int(n)
    return sum/np.size(tmp)
## END
mean_value = get_mean_for_bare_nulcei()
# mean_value  # 3.463519313304721<jupyter_output><empty_output><jupyter_text>We will replace the incompatible values with the mean values after converting them into int64.<jupyter_code>feature_vector["bare-nuclei"] = feature_vector["bare-nuclei"].replace('?', int(mean_value))<jupyter_output><empty_output><jupyter_text>Now we will change the type of the "bare-nuclei" feature to int64.<jupyter_code>feature_vector["bare-nuclei"] = feature_vector["bare-nuclei"].astype('int64')
feature_vector.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 699 entries, 0 to 698
Data columns (total 9 columns):
 #   Column                Non-Null Count  Dtype
---  ------                --------------  -----
 0   clump-thickness       699 non-null    int64
 1   cell-size             699 non-null    int64
 2   cell-shape            699 non-null    int64
 3   marginal-adhesion     699 non-null    int64
 4   epithelial-cell-size  699 non-null    int64
 5   bare-nuclei           699 non-null    int64
 6   bland-chromatin       699 non-null    int64
 7   normal-nucleoli       699 non-null    int64
 8   mitoses               699 non-null    int64
dtypes: int64(9)
memory usage: 49.3 KB<jupyter_text>Split the training and testing dataHere we will keep 30% of the entire dataset for testing.<jupyter_code>X_train, X_test, Y_train, Y_test = train_test_split(feature_vector, label_vector, test_size = 0.3, random_state = 2020)

print("shape of input training data:",X_train.shape)
print("shape of output training data:",Y_train.shape)
print("shape of input testing data:",X_test.shape)
print("shape of output testing data:",Y_test.shape)<jupyter_output>shape of input training data: (489, 9)
shape of output training data: (489,)
shape of input testing data: (210, 9)
shape of output testing data: (210,)<jupyter_text>Z-score Normalization<jupyter_code>sc = StandardScaler()
X_train_std = sc.fit_transform(X_train)
X_test_std = sc.transform(X_test)<jupyter_output><empty_output><jupyter_text>Train the model first<jupyter_code>def sigmoid_function(args):
    return (1/(1 + np.exp(-args)))
## END
def logLiklihood(z, y):
    """Log-liklihood function (cost function to be minimized in logistic regression classification)"""
    return -1 * np.sum((y * np.log(sigmoid_function(z))) + ((1 - y) * np.log(1 - sigmoid_function(z))))
## END
def model_optimize(w, b, X, Y):
    m = X.shape[0]
    #Prediction
    final_result = sigmoid_function(np.dot(w, X.T) + b)
    # Y_T = Y.T
    # cost = (-1/m)*(np.sum((Y_T*np.log(final_result)) + ((1 - Y_T)*(np.log(1 - final_result)))))
    cost = (-1/m)*(np.sum((np.asarray(Y.T)*np.log(final_result)) + ((1 - np.asarray(Y.T)*(np.log(1 - final_result))))))
    #Gradient calculation
    dw = (1/m)*(np.dot(X.T, (final_result - np.asarray(Y.T)).T))
    db = (1/m)*(np.sum(final_result - np.asarray(Y.T)))
    grads = {"dw": dw, "db": db}
    return grads, cost
## END
print("model_optimize")
def model_predict(w, b, X, Y, learning_rate, no_iterations):
    costs = []
    for i in range(no_iterations):
        grads, cost = model_optimize(w, b, X, Y)
        dw = grads["dw"]
        db = grads["db"]
        #weight update
        w = w - (learning_rate * (dw.T))
        b = b - (learning_rate * db)
        if (i % 100 == 0):
            costs.append(cost)
            print("Cost after %i iteration is %f" %(i, cost))
    #final parameters
    coeff = {"w": w, "b": b}
    gradient = {"dw": dw, "db": db}
    return coeff, gradient, costs
## END
print("model_predict")
def weightInitialization(n_features):
    w = np.zeros((1, n_features))
    b = 0
    return w, b
## END
print("weightInitialization")
def predict(final_pred, m):
    y_pred = np.zeros((1,m))
    for i in range(final_pred.shape[1]):
        if final_pred[0][i] > 0.5:
            y_pred[0][i] = 1
    return y_pred
## END
print("predict")
n_features = X_train_std.shape[1]
print('Number of Features', n_features)
w, b = weightInitialization(n_features)
coeff, gradient, costs = model_predict(w, b, X_train_std, Y_train, learning_rate = 0.01, no_iterations = 5001)
#Final prediction
w = coeff["w"]
b = coeff["b"]
print('Optimized weights', w)
print('Optimized intercept', b)
final_train_pred = sigmoid_function(np.dot(w, X_train_std.T) + b)
final_test_pred = sigmoid_function(np.dot(w, X_test_std.T) + b)
m_tr =  X_train_std.shape[0]
m_ts =  X_test_std.shape[0]
y_tr_pred = predict(final_train_pred, m_tr)
print('Training Accuracy', accuracy_score(y_tr_pred.T, Y_train))<jupyter_output>Training Accuracy 0.9652351738241309
{'fraction_non_alphanumeric': 0.11552423660046979, 'fraction_numerical': 0.02893444373265001, 'mean_word_length': 3.5360774818401937, 'pattern_counts': {'":': 6, '<': 59, '<?xml version=': 0, '>': 60, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '572751', 'license': 'MIT', 'n_tokens_mistral': 3312, 'n_tokens_neox': 3018, 'n_words': 864, 'path': 'classification-problems/logistic-regression-classifier-with-L2-regularization/breast-cancer-classifier-from-scratch.ipynb', 'repo_name': 'umang8496/machine-learning-hands-on', 'size': '9366'}