<jupyter_start><jupyter_text>Insight Data Science - Data Challenge \1<jupyter_code># Load required libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Loading the dataset Let's first load the Wisconsin Breast Cancer Dataset (previously converted to CSV format) downloaded from: https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Original)<jupyter_code>data = pd.read_csv('breast-cancer-wisconsin.csv',index_col=0)
data.head(6)<jupyter_output><empty_output><jupyter_text>Let's print a list of the columns and their corresponding datatypes in the dataset<jupyter_code>data.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 699 entries, 1000025 to 897471
Data columns (total 10 columns):
Clump Thickness                699 non-null int64
Uniformity of Cell Size        699 non-null int64
Uniformity of Cell Shape       699 non-null int64
Marginal Adhesion              699 non-null int64
Single Epithelial Cell Size    699 non-null int64
Bare Nuclei                    699 non-null object
Bland Chromatin                699 non-null int64
Normal Nucleoli                699 non-null int64
Mitoses                        699 non-null int64
Class                          699 non-null int64
dtypes: int64(9), object(1)
memory usage: 60.1+ KB<jupyter_text>We confirm that there are 699 instances in the dataset. Let's sort the dataset by ID<jupyter_code>data = data.sort_values(by=['ID'])
data.head(3)<jupyter_output><empty_output><jupyter_text>Data pre-processing Let's first, merely for convenience, change the labels for the categories "Class" column to:- **0 for benign**, and - **1 for malignant**,and see the result<jupyter_code>data['Class'] = data['Class'].replace(to_replace=2,value=0)
data['Class'] = data['Class'].replace(to_replace=4,value=1)

data.head(3)
data[data['Class']==1].count()<jupyter_output><empty_output><jupyter_text>According to the description of the dataset (see dataset summary table [here](https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Original) "UCI Machine Learning Repository")), there are 16 instances that contain a single missing (i.e., unavailable) attribute value, which is denoted by "?". Here we replace those missing values with NaN.<jupyter_code>data = data.replace(to_replace='?',value=np.nan)
data.head(3)<jupyter_output><empty_output><jupyter_text>We confirm by counting the number of `missing values` in the dataset. There are 16 instances in which the Bare Nuclei value is missing. This only represents ~2% of the entire dataset, so let's drop the rows with the missing values. Our dataset will now only have 683 rows.<jupyter_code>data = data.dropna(axis='index')
data['Bare Nuclei'] = data['Bare Nuclei'].astype(np.int64)
data.shape<jupyter_output><empty_output><jupyter_text>Exploratory Data Analysis (EDA)<jupyter_code>data.describe()
##data[data['Class']==0].hist(figsize=(15, 12),bins=10);
##data[data['Class']==1].hist(figsize=(15, 12),bins=10);

benign = data[data['Class']==0]
malignant = data[data['Class']==1]

#plt.figure(figsize=(10, 6))
fig, axs = plt.subplots(3, 3, figsize=(10, 10))
fig

#data.groupby("Class")[].plot(kind='kde', ax=axs[1])
data.groupby("Class")['Clump Thickness'].hist(alpha=0.4, ax=axs[0, 0])
axs[0,0].set_title('Clump Thickness')
data.groupby("Class")['Uniformity of Cell Size'].hist(alpha=0.4, ax=axs[0, 1])
axs[0,1].set_title('Uniformity of Cell Size')
data.groupby("Class")['Uniformity of Cell Shape'].hist(alpha=0.4, ax=axs[0, 2])
axs[0,2].set_title('Uniformity of Cell Shape')
data.groupby("Class")['Marginal Adhesion'].hist(alpha=0.4, ax=axs[1, 0])
axs[1,0].set_title('Marginal Adhesion')
data.groupby("Class")['Single Epithelial Cell Size'].hist(alpha=0.4, ax=axs[1, 1])
axs[1,1].set_title('Single Epithelial Cell Size')
data.groupby("Class")['Bare Nuclei'].hist(alpha=0.4, ax=axs[1, 2])
axs[1,2].set_title('Bare Nuclei')
data.groupby("Class")['Bland Chromatin'].hist(alpha=0.4, ax=axs[2, 0])
axs[2,0].set_title('Bland Chromatin')
data.groupby("Class")['Normal Nucleoli'].hist(alpha=0.4, ax=axs[2, 1])
axs[2,1].set_title('Normal Nucleoli')
data.groupby("Class")['Mitoses'].hist(alpha=0.4, ax=axs[2, 2])
axs[2,2].set_title('Mitoses')


#data.Class

data['Class'].hist(alpha=0.4)
sns.pairplot(data=data, hue="Class", palette="Set2", diag_kind="hist", height=4.5);
plt.figure(figsize=(5, 5));
plt.matshow(data.corr(),cmap='afmhot');
plt.colorbar();<jupyter_output><empty_output><jupyter_text>Modelling<jupyter_code># Load sklearn for data modelling
from sklearn.model_selection import train_test_split, cross_validate, KFold
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.metrics import classification_report, confusion_matrix, roc_auc_score, roc_curve<jupyter_output><empty_output><jupyter_text>Let's first split the data into two datasets for Training and Testing<jupyter_code>X = data.loc[:, data.columns != 'Class']
y = data.loc[:, data.columns == 'Class']

#X = data.iloc[:,:-1], 
#y = data.iloc[:,-1]

X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, test_size=0.2, random_state=42)
scaler = StandardScaler()
scaler.fit(X_train.astype('float64'))

X_train = scaler.transform(X_train.astype('float64'))  
X_test = scaler.transform(X_test.astype('float64'))<jupyter_output><empty_output><jupyter_text>Logistic Regression model<jupyter_code>logreg = LogisticRegression(C=1e6,solver='lbfgs',multi_class='auto')
logreg.fit(X_train, y_train.values.ravel())

y_pred = logreg.predict(X_test)
print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))
print('Accuracy of Logistic regression classifier on training set: {:.2f}'
     .format(logreg.score(X_train, y_train)))
print('Accuracy of Logistic regression classifier on test set: {:.2f}'
     .format(logreg.score(X_test, y_test)))
logit_roc_auc = roc_auc_score(y_test, logreg.predict(X_test))
fpr, tpr, thresholds = roc_curve(y_test, logreg.predict_proba(X_test)[:,1])
plt.figure()
plt.plot(fpr, tpr, label='Logistic Regression (area = %0.2f)' % logit_roc_auc)
plt.plot([0, 1], [0, 1],'r--')
plt.xlim([-0.015, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver operating characteristic')
plt.legend(loc="lower right")
plt.savefig('Log_ROC')
plt.show()
kfold = KFold(n_splits=5, random_state=42, shuffle=True)
cross_validate(logreg, X, y.values.ravel(), cv=kfold, 
               scoring=('accuracy','precision', 'recall', 'roc_auc'),return_train_score=True)<jupyter_output><empty_output><jupyter_text>Decision Trees<jupyter_code>trees = DecisionTreeClassifier()
trees.fit(X_train, y_train.values.ravel())

y_pred = trees.predict(X_test)
print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))

print('Accuracy of Decision Tree classifier on training set: {:.2f}'
     .format(trees.score(X_train, y_train)))
print('Accuracy of Decision Tree classifier on test set: {:.2f}'
     .format(trees.score(X_test, y_test)))<jupyter_output>[[89  0]
 [ 4 44]]
              precision    recall  f1-score   support

           0       0.96      1.00      0.98        89
           1       1.00      0.92      0.96        48

   micro avg       0.97      0.97      0.97       137
   macro avg       0.98      0.96      0.97       137
weighted avg       0.97      0.97      0.97       137

Accuracy of Decision Tree classifier on training set: 1.00
Accuracy of Decision Tree classifier on test set: 0.97<jupyter_text>Gaussian Naive Bayes model<jupyter_code>GaussNB = GaussianNB(priors=None);
GaussNB.fit(X_train, y_train.values.ravel());

y_pred = GaussNB.predict(X_test)
print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))
print('Accuracy of GaussianNB classifier on test set: {:.2f}'.format(GaussNB.score(X_test, y_test)))<jupyter_output>[[88  1]
 [ 4 44]]
              precision    recall  f1-score   support

           0       0.96      0.99      0.97        89
           1       0.98      0.92      0.95        48

   micro avg       0.96      0.96      0.96       137
   macro avg       0.97      0.95      0.96       137
weighted avg       0.96      0.96      0.96       137

Accuracy of GaussianNB classifier on test set: 0.96<jupyter_text>K Nearest Neighbors model<jupyter_code>kNN = KNeighborsClassifier(n_neighbors=3)  
kNN.fit(X_train, y_train.values.ravel());

y_pred = kNN.predict(X_test)

print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))
print('Accuracy of kNN classifier on test set: {:.2f}'.format(kNN.score(X_test, y_test)))
error = []

# Calculating error for K values between 1 and 40
for i in range(1, 15):  
    knn = KNeighborsClassifier(n_neighbors=i)
    knn.fit(X_train, y_train.values.ravel())
    pred_i = knn.predict(X_test)
    error.append(np.mean(pred_i != y_test.values.ravel()))

plt.figure(figsize=(12, 6))  
plt.plot(range(1, 15), error, color='red', linestyle='dashed', marker='o',  
         markerfacecolor='blue', markersize=10)
plt.title('Error Rate K Value')  
plt.xlabel('K Value')
plt.ylabel('Mean Error')
error.index(min(error))<jupyter_output><empty_output><jupyter_text>SVM<jupyter_code>svm = SVC()
svm.fit(X_train, y_train.values.ravel())


y_pred = svm.predict(X_test)

print(confusion_matrix(y_test, y_pred))
print(classification_report(y_test, y_pred))
print('Accuracy of SVM classifier on training set: {:.2f}'
     .format(svm.score(X_train, y_train.values.ravel())))
print('Accuracy of SVM classifier on test set: {:.2f}'
     .format(svm.score(X_test, y_test.values.ravel())))<jupyter_output>[[88  1]
 [ 4 44]]
              precision    recall  f1-score   support

           0       0.96      0.99      0.97        89
           1       0.98      0.92      0.95        48

   micro avg       0.96      0.96      0.96       137
   macro avg       0.97      0.95      0.96       137
weighted avg       0.96      0.96      0.96       137

Accuracy of SVM classifier on training set: 0.98
Accuracy of SVM classifier on test set: 0.96
{'fraction_non_alphanumeric': 0.13660618996798293, 'fraction_numerical': 0.044823906083244394, 'mean_word_length': 4.456855479089465, 'pattern_counts': {'":': 0, '<': 58, '<?xml version=': 0, '>': 58, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '380521', 'license': 'MIT', 'n_tokens_mistral': 4040, 'n_tokens_neox': 3652, 'n_words': 824, 'path': 'Challenge01/challenge01.ipynb', 'repo_name': 'Alex-Preciado/Data-Challenges', 'size': '10307'}