<jupyter_start><jupyter_text>PCA of a breast cancer data setIn the following we will look at a breast cancer data set comprising of real-valued multivariate data. The data decomposes into two classes representing if a patient has breast cancer or not.The class representing the malignant cases has 212 samples. The benign cases comprise of 357 samples.The data itself is of dimension 30, i.e. each case (malignant or benign) has 30 features indicating the radius, texture, area,... of the sample.The dataset used hereafter can be obtained from the following link:https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic)The discussion itself is inspired by the tutorial:https://www.datacamp.com/community/tutorials/principal-component-analysis-in-pythonFor resources on the topic of principal component analysis itself, we refer to the following links:1) https://www.nature.com/articles/nmeth.43462) https://www.youtube.com/watch?v=kuzJJgPBrqc3) https://en.wikipedia.org/wiki/Principal_component_analysis  1) Data preprocessing  Loading necessary packages<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Loading the datasetWe load the dataset (without the labels) and print the shape which confirms that it consists of 569 samples with 30 features.<jupyter_code>from sklearn.datasets import load_breast_cancer
breast = load_breast_cancer()
breast_data = breast.data
breast_data.shape<jupyter_output><empty_output><jupyter_text>Each sample has a label attached. We load the labels, compare the shapes and reshape the labels array for further processing:<jupyter_code>breast_labels = breast.target
breast_labels.shape
labels = np.reshape(breast_labels,(569,1))<jupyter_output><empty_output><jupyter_text>After reshaping both arrays (data and labels) can be concatenated:<jupyter_code>final_breast_data = np.concatenate([breast_data,labels],axis=1)
final_breast_data.shape<jupyter_output><empty_output><jupyter_text>From the concatenated arrays we create a dataframe to have the information ready in tabular form:<jupyter_code>breast_dataset = pd.DataFrame(final_breast_data)<jupyter_output><empty_output><jupyter_text>In the next step we print the 30 features of the dataset and will then add the label field:<jupyter_code>features = breast.feature_names
features
features_labels = np.append(features,'label')<jupyter_output><empty_output><jupyter_text>In the next step we allocate the column names to the dataframe and print its head afterwards:<jupyter_code>breast_dataset.columns = features_labels
breast_dataset.head()<jupyter_output><empty_output><jupyter_text>The original labels are changed to benign and malignant:<jupyter_code>breast_dataset['label'].replace(0, 'Benign',inplace=True)
breast_dataset['label'].replace(1, 'Malignant',inplace=True)<jupyter_output><empty_output><jupyter_text>We print the tail of the dataset to demonstrate that the labels have just been changed:<jupyter_code>breast_dataset.tail()<jupyter_output><empty_output><jupyter_text>2) Data visualization using PCA We start off by adjusting the data across all dimensions to fit onto the same scale. The normalization is applied to the features data and assumes that it is normally distributed. Its result is a distribution  with zero mean and unit standard deviation.<jupyter_code>from sklearn.preprocessing import StandardScaler
x = breast_dataset.loc[:, features].values
x = StandardScaler().fit_transform(x)
x.shape
np.mean(x),np.std(x)<jupyter_output><empty_output><jupyter_text>In the following the normalized features are brought into tabular form:<jupyter_code>feat_cols = ['feature'+str(i) for i in range(x.shape[1])]
normalised_breast = pd.DataFrame(x,columns=feat_cols)
normalised_breast.tail()<jupyter_output><empty_output><jupyter_text>The most important step consists of the PCA itself. In the following we will project the 30 dimensional data to three principal components.<jupyter_code>from sklearn.decomposition import PCA
pca_breast = PCA(n_components=3)
principalComponents_breast = pca_breast.fit_transform(x)<jupyter_output><empty_output><jupyter_text>In the next step we build a dataframe which only has the just produced principal component values for all of the 569 samples:<jupyter_code>principal_breast_Df = pd.DataFrame(data = principalComponents_breast
             , columns = ['principal component 1', 'principal component 2','principal component 3'])<jupyter_output><empty_output><jupyter_text>We print head and tail of the new dataframe:<jupyter_code>principal_breast_Df.head()
principal_breast_Df.tail()<jupyter_output><empty_output><jupyter_text>With the principal compoments we can give the so-called explained variance ratio. It is a measure of variance (thus information) telling us how much variance a principal component holds.<jupyter_code>print('Explained variance per principal component: {}'.format(pca_breast.explained_variance_ratio_))<jupyter_output>Explained variance per principal component: [0.44272026 0.18971182 0.09393163]<jupyter_text>From these numbers we notice that while projecting from 30 dimensions to 3d data about 28% of the information is lost. In the following we produce a visualization of the 569 samples along the axes provided by the 1st and 2nd principal components. From this we can retrieve the sample distribution along the two label classes.<jupyter_code>#setting up the plot
plt.figure()
plt.figure(figsize=(10,10))
plt.xticks(fontsize=12)
plt.yticks(fontsize=14)
plt.xlabel('1st principal component',fontsize=20)
plt.ylabel('2nd principal component',fontsize=20)
plt.title("PCA of a breast cancer dataset",fontsize=20)
targets = ['Benign', 'Malignant']
colors = ['m', 'b']

#insertion of the data into the plot
for target, color in zip(targets,colors):
    indicesToKeep = breast_dataset['label'] == target
    plt.scatter(principal_breast_Df.loc[indicesToKeep, 'principal component 1']
               , principal_breast_Df.loc[indicesToKeep, 'principal component 2'], c = color, s = 50)

#setting up the legend
plt.legend(targets,prop={'size': 15})

plt.show()<jupyter_output><empty_output><jupyter_text>From this 2d plot we gain two insights. First, we see that the two label classes are (mostly) linearily separable. Second, malignant class is less spread out as compared to the other one. We also realize that when using the 1st and 3rd or 2nd and 3rd pcs for plotting that the separation is less pronounced. Hence, using the the 1st and 2nd pcs is most profitable for further information extraction. This is further supported by plotting the samples over the three first principal components:<jupyter_code>targets = ['Benign', 'Malignant']
colors = ['m', 'b']

fig = plt.figure(figsize = (15,6))

ax = fig.add_subplot(111, projection = '3d')
ax.title.set_text("PCA of a breast cancer dataset")
ax.set_xlabel('1st principal component',fontsize=15)
ax.set_ylabel('2nd principal component',fontsize=15)
ax.set_zlabel('3rd principal component',fontsize=15)

for target, color in zip(targets,colors):
    indicesToKeep = breast_dataset['label'] == target
    ax.scatter(principal_breast_Df.loc[indicesToKeep, 'principal component 1']
               , principal_breast_Df.loc[indicesToKeep, 'principal component 2'],principal_breast_Df.loc[indicesToKeep, 'principal component 3'],
          c = color, s = 20)

plt.legend(targets,prop={'size': 10})
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08522650893937357, 'fraction_numerical': 0.01774432047318188, 'mean_word_length': 6.774294670846395, 'pattern_counts': {'":': 0, '<': 68, '<?xml version=': 0, '>': 68, 'https://': 5, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '105785', 'license': 'MIT', 'n_tokens_mistral': 2353, 'n_tokens_neox': 2191, 'n_words': 713, 'path': 'PCA - breast cancer data set.ipynb', 'repo_name': 'andreaspts/ML---PCAs', 'size': '7439'}