<jupyter_start><jupyter_text>Your Turn!In the last video, you saw two of the main aspects of principal components:1. **The amount of variability captured by the component.**2. **The components themselves.**In this notebook, you will get a chance to explore these a bit more yourself.  First, let's read in the necessary libraries, as well as the data.<jupyter_code>import pandas as pd
import numpy as np
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, accuracy_score
from helper_functions import show_images, do_pca, scree_plot, plot_component
import test_code as t

import matplotlib.image as mpimg
import matplotlib.pyplot as plt
import seaborn as sns

%matplotlib inline

#read in our dataset
train = pd.read_csv('./data/train.csv')
train.fillna(0, inplace=True)

# save the labels to a Pandas series target
y = train['label']
# Drop the label feature
X = train.drop("label",axis=1)

show_images(30)<jupyter_output>/home/workspace/helper_functions.py:44: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
  mat_data = X.iloc[digit_num].as_matrix().reshape(28,28)  #reshape images<jupyter_text>`1.` Perform PCA on the **X** matrix using on your own or using the **do_pca** function from the **helper_functions** module. Reduce the original more than 700 features to only 10 principal components.<jupyter_code># Make sure to keep track of the resulting components and the pca object
do_pca?
pca, X_pca = do_pca(10, X)<jupyter_output><empty_output><jupyter_text>`2.` Now use the **scree_plot** function from the **helper_functions** module to take a closer look at the results of your analysis.<jupyter_code>scree_plot(pca)<jupyter_output><empty_output><jupyter_text>`3.` Using the results of your scree plot, match each letter as the value to the correct key in the **solution_three** dictionary.  Once you are confident in your solution run the next cell to see if your solution matches ours.<jupyter_code>a = True
b = False
c = 6.13
d = 'The total amount of variability in the data explained by the first two principal components'
e = None

solution_three = {
    '10.42' : d, 
    'The first component will ALWAYS have the most amount of variability explained.': a,
    'The total amount of variability in the data explained by the first component': c,
    'The sum of the variability explained by all the components can be greater than 100%': b
}
#Run this cell to see if your solution matches ours
t.question_3_check(solution_three)<jupyter_output>Looks good!  The amount of variability explained by each principal component gives us an idea of how much of the original variability in the original data is retained by each component.  Nice job matching these up!<jupyter_text>`4.` Use the **plot_component** function from the **helper_functions** module to look at each of the components (remember they are 0 indexed).  Use the results to assist with question 5.<jupyter_code># Use the images of the component weights to answer the next question
plot_component(pca, 0)<jupyter_output><empty_output><jupyter_text>`5.` Using the results from viewing each of your principal component weights in question 4, change the following values of the **solution_five** dictionary to the **number of the index** for the principal component that best matches the description.  Once you are confident in your solution run the next cell to see if your solution matches ours.<jupyter_code>solution_five = {
    'This component looks like it will assist in identifying zero': 0,
    'This component looks like it will assist in identifying three': 3
}
#Run this cell to see if your solution matches ours
t.question_5_check(solution_five)<jupyter_output>Nice job!  That matches our solution as well!  The index of the first principal component appears to have really high weights where a zero would appear.  Alternatively, the fourth (third indexed component) appears to downweight where a three would appear to make it stand out.
{'fraction_non_alphanumeric': 0.06533269045323047, 'fraction_numerical': 0.010125361620057859, 'mean_word_length': 5.295902883156297, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '473767', 'license': 'MIT', 'n_tokens_mistral': 1180, 'n_tokens_neox': 1117, 'n_words': 544, 'path': 'Unsupervised_Learning/4. Dimensionality_Reduction_and_PCA/Interpret_PCA_Results.ipynb', 'repo_name': 'annadutkiewicz/Intro_ML_Tensorflow', 'size': '4148'}