<jupyter_start><jupyter_text>**Principal Component Analysis (PCA)**An *Unsupervised Statistical Learning Technique* used to examine the interrelations among a set of variables.- Where Regression Analysis determines the best line of a data fit, Factor Analysis determines several orthogonal lines of best fit to a data set.- Orthogonal means 'at right angles'1. actually the lines are perpendicular to each other in N-dimensional space- n-Dimensional space is the variable sample space1. There are as many dimensions as there are variables. **In a dataset with 4 variables the sample space is 4-Dimensional**Example of an orthogonal fit to a best fit:![Orthogonal fit to line of best fit](https://4.bp.blogspot.com/-urn6_TX82iE/VGetWlAH-aI/AAAAAAAACns/CFNvruaOb4A/s1600/Capture.GIF)**Components**- Components are a linear transformation that chooses a variable system for the data set such that the greatest variance of the dataset comes to lie on the first axes- then the Second Greatest variance on the second axes- and so on- The process allows us to reduce the number of variables used in an analysis- The components are uncorrelated, since in the sample space they are orthogonal (at direct right angles) to each otherWe can continue this analysis into higher dimensionsComponent 1 explains most of the variance. Component 2 explaining the next largest chunk, Component 3 the next largest chunk, etc*If we use this technique on a dataset with a large number of variables, we can compress the amount of explained variation down to a few components*. Then we want to get rid of the components that don't explain much of the variance of the data.The most challenging part of PCA is interpreting the components.Typically with PCA you want to *Standardize* the data.This is only used for Analysis of data and is not a fully deployable model<jupyter_code># Import the Cancer dataset
from sklearn.datasets import load_breast_cancer
# load the data
cancer = load_breast_cancer()
# "Bunch" essentially acts like a dictionary
type(cancer)
# Get keys
cancer.keys()
# get description with DESCR. The data is kept in data and features in feature_names
print(cancer['DESCR'])<jupyter_output>.. _breast_cancer_dataset:

Breast cancer wisconsin (diagnostic) dataset
--------------------------------------------

**Data Set Characteristics:**

    :Number of Instances: 569

    :Number of Attributes: 30 numeric, predictive attributes and the class

    :Attribute Information:
        - radius (mean of distances from center to points on the perimeter)
        - texture (standard deviation of gray-scale values)
        - perimeter
        - area
        - smoothness (local variation in radius lengths)
        - compactness (perimeter^2 / area - 1.0)
        - concavity (severity of concave portions of the contour)
        - concave points (number of concave portions of the contour)
        - symmetry 
        - fractal dimension ("coastline approximation" - 1)

        The mean, standard error, and "worst" or largest (mean of the three
        largest values) of these features were computed for each image,
        resulting in 30 features.  For instance, field 3 is Mean Radius, f[...]<jupyter_text>**A small number of instances (569) but a relatively large number of attributes (30)** Breast cancer wisconsin (diagnostic) dataset:--------------------------------------------**Data Set Characteristics:**    :Number of Instances: 569    :Number of Attributes: 30 numeric, predictive attributes and the classIf we were trying to classify the data, we'd want to try to predict if they are malignant or benign. *In this case we want to find **what components are the Most Important** (Cause the most variance) in the dataset*If you're trying to perform Classification on a dataset, it makes sense to run PCA to get a sense of what's most Important.<jupyter_code># set up the dataframe
df = pd.DataFrame(cancer['data'],columns=cancer['feature_names'])
df.info()
df.head()
cancer.target_names<jupyter_output><empty_output><jupyter_text>It's difficult to visualize High Dimensional Data. With PCA we'll find the First Two Principal Components.<jupyter_code># import the model
from sklearn.preprocessing import StandardScaler<jupyter_output><empty_output><jupyter_text>**Scale the data so that it has a single unit variance before we apply the PCA** What we're doing - instantiate the PCA object- find the principal components using the fit method- apply the rotation and dimensionality reduction by calling transform<jupyter_code># Instantiate the Scaler object
scaler = StandardScaler()
# Fit the dataframe
scaler.fit(df)
# Now transform the data
scaled_data = scaler.transform(df)<jupyter_output><empty_output><jupyter_text>We can also specify how many components we want to keep using the PCA object Performing the PCA<jupyter_code># import PCA
from sklearn.decomposition import PCA
# Only use the top 2 components to start
pca = PCA(n_components=2)
# Fit the pca to the scaled data
pca.fit(scaled_data)
# Transform the data to the first principal components
x_pca = pca.transform(scaled_data)
# The original data shape is 569, 30
scaled_data.shape
# The new data shape is 569, 2 (no, rows by n_components)
x_pca.shape<jupyter_output><empty_output><jupyter_text>The data is not easy to make sense of, but it's still a powerful technique for driving down complexity.Using c, cmap and alpha see how the difference between malignant or benign stands out. We're able to make more sense of of the data with 2 columns, rather than digging through 30 columns<jupyter_code># Plot these out
# Take all the rows from column 0 and plot those against all the rows in column 1    
plt.figure(figsize=(8,6))
plt.scatter(x_pca[:,0],x_pca[:,1],c=cancer['target'],cmap='rainbow',alpha=0.5)
plt.xlabel('First Principal Component')
plt.ylabel('Second Principal Component')<jupyter_output><empty_output><jupyter_text>The components correspond to *Combinations* of the original features. The components themselves are stored as *Attributes of the PCA object*Each Row corresponds to the Component and each Column maps back to the original Features<jupyter_code>pca.components_<jupyter_output><empty_output><jupyter_text>Example this relationship with a heatmap<jupyter_code># create a dataframe with both the components and the original feature names
df_comp = pd.DataFrame(pca.components_,columns=cancer['feature_names'])
df_comp.head()<jupyter_output><empty_output><jupyter_text>**Each row is a Component and each Column is a Feature. The higher the number (hotter or brighter the color), the more correlated it is.**<jupyter_code>plt.figure(figsize=(14,7))
sns.heatmap(df_comp,cmap='plasma')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07820895522388059, 'fraction_numerical': 0.009402985074626866, 'mean_word_length': 4.659628378378378, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 36, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '160382', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 1880, 'n_tokens_neox': 1729, 'n_words': 834, 'path': 'Analyzing Breast Cancer using Principal Component Analysis.ipynb', 'repo_name': 'jsutch/Principal_Component_Analysis', 'size': '6700'}