<jupyter_start><jupyter_text>Classifying Breast Cancer Dataset - PCA for feature fusionProblem: Is practicable to use PCA with this dataset?Link: https://archive.ics.uci.edu/ml/datasets/Breast+Cancer 1. Collect dataData is standardized:a) Interval variables were convert to yours mean.b) Missing values were kept to zero.<jupyter_code>import pandas as pd

# Open the file and read it as table
file = open("dataset-full2.csv", "r")

df = pd.read_csv(file, names=['Class', 'Age', 'Age mean', 'Menopause','Tumor-size', 'Tumor-size mean', 'Inv-nodes', 'Inv-nodes mean', 'Node-caps', 'Deg-malig', 'Breast', 'Breast-quad', 'Irradiat'])
df<jupyter_output><empty_output><jupyter_text>2. Normality TestTest to mean variables.<jupyter_code>from scipy import stats

mean_variables = ['Age mean', 'Tumor-size mean', 'Inv-nodes mean']
normal_variables = list()

for variable in mean_variables:
    var_test = df[variable]
    k2, p = stats.normaltest(var_test)
    alpha = 1e-3
    print("Variable: %s" % (variable))
    print("p = %s" % (p))
    if p < alpha:  # null hypothesis: x comes from a normal distribution
        print("The null hypothesis can be rejected\n")
    else:
        print("The null hypothesis cannot be rejected")
        print("Feature comes from a normal distribution\n")
        normal_variables.append(variable)
print("Normal variables: %s" % (normal_variables))<jupyter_output>Variable: Age mean
p = 0.00944546987698
The null hypothesis cannot be rejected
Feature comes from a normal distribution

Variable: Tumor-size mean
p = 0.529472683757
The null hypothesis cannot be rejected
Feature comes from a normal distribution

Variable: Inv-nodes mean
p = 7.47107744563e-44
The null hypothesis can be rejected

Normal variables: ['Age mean', 'Tumor-size mean']<jupyter_text>3. Choose features (using PCA)2-dimensional features (Principal Components = 2)Used to visualization.<jupyter_code>from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA

# Separating out the features
x = df.loc[:, normal_variables].values
#x = StandardScaler().fit_transform(x) # Normal distribution standardization

pca = PCA(n_components=2)
principalComponents = pca.fit_transform(x)

principalDf = pd.DataFrame(data = principalComponents
             , columns = ['principal component 1', 'principal component 2'])

# Create a matrix with class column
finalDf = pd.concat([principalDf, df[['Class']]], axis = 1)
print(finalDf)

print("Variance ratios: %s" % (pca.explained_variance_ratio_))
print("Total: %s" % sum(pca.explained_variance_ratio_))<jupyter_output>principal component 1  principal component 2                 Class
0                -7.888727             -15.467630  no-recurrence-events
1                 3.761520              -7.450663  no-recurrence-events
2                 3.761520              -7.450663  no-recurrence-events
3                12.311602              11.308230  no-recurrence-events
4                23.428733             -11.083943  no-recurrence-events
5                12.311602              11.308230  no-recurrence-events
6                 0.661356               3.291263  no-recurrence-events
7                 7.394799              12.216550  no-recurrence-events
8               -25.739300              -2.000744  no-recurrence-events
9                 3.761520              -7.450663  no-recurrence-events
10               23.428733             -11.083943  no-recurrence-events
11                0.661356               3.291263  no-recurrence-events
12               17.228406              10.399910  no-recurrence[...]<jupyter_text>4. Plot without PCA vs. with PCA Projection of components after orthogonal transformation.<jupyter_code>import matplotlib.pyplot as plt

fig = plt.figure(figsize=(10,10))
ax = fig.add_subplot(111)
ax.set_xlabel(normal_variables[0], fontsize = 15)
ax.set_ylabel(normal_variables[1], fontsize = 15)
ax.set_title('Classes Plot (without PCA))', fontsize = 20)

targets = ['recurrence-events', 'no-recurrence-events']
class1 = df.loc[df['Class'] == targets[1]].values
class2 = df.loc[df['Class'] == targets[0]].values
plt.scatter(class2[0:, 2], class2[0:, 5]) # Age mean
plt.scatter(class1[0:, 2], class1[0:, 5]) # Tumor-size mean
ax.legend(targets)
ax.grid()
plt.show();

fig = plt.figure(figsize=(10,10))
ax = fig.add_subplot(111) 
ax.set_xlabel('Principal Component 1', fontsize = 15)
ax.set_ylabel('Principal Component 2', fontsize = 15)
ax.set_title('2 components PCA', fontsize = 20)
targets = ['no-recurrence-events', 'recurrence-events']
colors = ['r', 'g', 'b']
for target, color in zip(targets,colors):
    indicesToKeep = finalDf['Class'] == target
    ax.scatter(  finalDf.loc[indicesToKeep, 'principal component 1']
               , finalDf.loc[indicesToKeep, 'principal component 2']
               , c = color
               , s = 50)
ax.legend(targets)
ax.grid()
plt.show();<jupyter_output><empty_output><jupyter_text>5. Get the correlation matrixMean variables from dataset only.Features with very little correlation.<jupyter_code>dfc = df[['Age mean', 'Tumor-size mean', 'Inv-nodes mean']] 
dfc.corr().values<jupyter_output><empty_output><jupyter_text>6. Bonus: PCA with 1 Principal Component1-dimensional features.Used to fusion normal variables.<jupyter_code>import numpy as np

x = df.loc[:, normal_variables].values
pca = PCA(n_components=1)
principalComponents = pca.fit_transform(x)

principalDf = pd.DataFrame(data = principalComponents
             , columns = ['principal component 1'])
finalDf = pd.concat([principalDf, df[['Class']]], axis = 1)
import matplotlib.pyplot as pp
val = 0. # this is the value where you want the data to appear on the y-axis.
ar = principalDf['principal component 1'] # just as an example array
ax=plt.subplot(111)
ax.set_title('1 component PCA')
ax.set_xlabel('principal component 1')
pp.plot(ar, np.zeros_like(ar) + val, 'x')
pp.show()
print("Variance ratios: %s" % (pca.explained_variance_ratio_))
print("Total: %s" % sum(pca.explained_variance_ratio_))<jupyter_output><empty_output><jupyter_text>7. Bonus: PCA with 2 Principal Components (mean variables)2-dimensional features (numeric variables only).Used to fusion normal variables.<jupyter_code># Separating out the features
x = df.loc[:, mean_variables].values

pca = PCA(n_components=2)
principalComponents = pca.fit_transform(x)

principalDf = pd.DataFrame(data = principalComponents
             , columns = ['principal component 1', 'principal component 2'])

# Create a matrix with class column
finalDf = pd.concat([principalDf, df[['Class']]], axis = 1)

print("Variance ratios: %s" % (pca.explained_variance_ratio_))
print("Total: %s" % sum(pca.explained_variance_ratio_))

fig = plt.figure(figsize=(10,10))
ax = fig.add_subplot(111) 
ax.set_xlabel('Principal Component 1', fontsize = 15)
ax.set_ylabel('Principal Component 2', fontsize = 15)
ax.set_title('2 components PCA', fontsize = 20)
targets = ['no-recurrence-events', 'recurrence-events']
colors = ['r', 'g', 'b']
for target, color in zip(targets,colors):
    indicesToKeep = finalDf['Class'] == target
    ax.scatter(  finalDf.loc[indicesToKeep, 'principal component 1']
               , finalDf.loc[indicesToKeep, 'principal component 2']
               , c = color
               , s = 50)
ax.legend(targets)
ax.grid()
plt.show();<jupyter_output>Variance ratios: [ 0.50293327  0.44603061]
Total: 0.948963877209<jupyter_text>8. Bonus: PCA with 3 Principal Components (mean variables)3-dimensional features (numeric variables only).Used to projection of components after orthogonal transformation.<jupyter_code>from mpl_toolkits.mplot3d.axes3d import Axes3D

x = df.loc[:, mean_variables].values
pca = PCA(n_components=3)
principalComponents = pca.fit_transform(x)

principalDf = pd.DataFrame(data = principalComponents
             , columns = ['principal component 1', 'principal component 2', 'principal component 3'])
finalDf = pd.concat([principalDf, df[['Class']]], axis = 1)

print("Variance ratios: %s" % (pca.explained_variance_ratio_))
print("Total: %s" % sum(pca.explained_variance_ratio_))

# Create the 3d chart
fig = plt.figure(figsize= (10,10))
ax = fig.add_subplot(111, projection='3d') 
ax.set_xlabel('Principal Component 1', fontsize = 15)
ax.set_ylabel('Principal Component 2', fontsize = 15)
ax.set_zlabel('Principal Component 3', fontsize = 15)
ax.set_title('3 components PCA', fontsize = 20)
targets = ['no-recurrence-events', 'recurrence-events']
colors = ['r', 'g', 'b']
for target, color in zip(targets,colors):
    indicesToKeep = finalDf['Class'] == target
    ax.scatter(  finalDf.loc[indicesToKeep, 'principal component 1']
               , finalDf.loc[indicesToKeep, 'principal component 2']
               , finalDf.loc[indicesToKeep, 'principal component 3']
               , c = color
               , s = 50)
ax.legend(targets)
ax.grid()
plt.show()<jupyter_output>Variance ratios: [ 0.50293327  0.44603061  0.05103612]
Total: 1.0
{'fraction_non_alphanumeric': 0.12431464697325724, 'fraction_numerical': 0.051247622244601096, 'mean_word_length': 3.8681917211328978, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 29, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '561869', 'license': 'MIT', 'n_tokens_mistral': 3258, 'n_tokens_neox': 2916, 'n_words': 749, 'path': 'PCA_ByAnd.ipynb', 'repo_name': 'Andygrammer/Pattern-Recognition-Breast-Cancer', 'size': '8937'}