<jupyter_start><jupyter_text>Import Data<jupyter_code>from sklearn.datasets import load_breast_cancer
cancer_dictionary = load_breast_cancer()
cancer_dictionary.keys()
df = pd.DataFrame(cancer_dictionary['data'], columns=cancer_dictionary['feature_names'])
df.head()<jupyter_output><empty_output><jupyter_text>1) Scale the data<jupyter_code>from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaled_X = scaler.fit_transform(df)
scaled_X
scaled_X.mean()<jupyter_output><empty_output><jupyter_text>2) PCA with sklearn<jupyter_code>from sklearn.decomposition import PCA
# help(PCA)
pca_model = PCA(n_components=2)<jupyter_output><empty_output><jupyter_text>Sklearn will do the following:+ Fitting will find Eigen Values and Eigen Vectors+ Transorm will project those values (making dot product) on the original data<jupyter_code>pca_results = pca_model.fit_transform(scaled_X) 
pca_results
plt.figure(figsize=(8,6), dpi=150)
plt.scatter(pca_results[:, 0], pca_results[:, 1], c=cancer_dictionary['target']);<jupyter_output><empty_output><jupyter_text>------  3) Principal Components of the model<jupyter_code>pca_model.components_
df_comp = pd.DataFrame(pca_model.components_, index=['PC1', 'PC2'], columns=df.columns)
df_comp.head()
plt.figure(figsize=(20,3), dpi=200)
sns.heatmap(df_comp, annot=True);<jupyter_output><empty_output><jupyter_text>4) Explained Variance Ratio This means first principal component explains 44.27% variance of the data and second one explains around 19%.Both components can explain 63% of the variance of the dataset. We lose around 37% of the explained variance.<jupyter_code>pca_model.explained_variance_ratio_
np.sum(pca_model.explained_variance_ratio_)<jupyter_output><empty_output><jupyter_text>---------  ExpirementationIf choosing 2 Principal Component can explain 63% of variance of data and lose around 37%.What if we increase the number of Principal Component, what will happen?<jupyter_code>pca_30 = PCA(n_components=30)
pca_30.fit(scaled_X) # we don't really need to transform here as we are only interested to find out ratio.
pca_30.explained_variance_ratio_
np.sum(pca_30.explained_variance_ratio_)<jupyter_output><empty_output><jupyter_text>Above value 1.0 makes scense. Because we are using 30 principal components out of original 30 features. Of course, we didn't lose any explained variance at all. As a result, the value is 1.0 (100%).  Let's find out for various number of Principal Components<jupyter_code>len(scaled_X[0])
explained_variance = []

for n in range(1, len(scaled_X[0])):
    pca_model = PCA(n_components=n)
    pca_model.fit(scaled_X)
    
    variance_ratio_total = np.sum(pca_model.explained_variance_ratio_)
    explained_variance.append(variance_ratio_total)<jupyter_output><empty_output><jupyter_text>Using this elbow method, we can figure out what is the best `n_componet` to choose.<jupyter_code>plt.plot(range(1, len(scaled_X[0])), explained_variance)
plt.xlabel('Number of Components')
plt.ylabel('Variance Explained');<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10220177456457444, 'fraction_numerical': 0.020046007229707525, 'mean_word_length': 7.848837209302325, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '634574', 'license': 'MIT', 'n_tokens_mistral': 1100, 'n_tokens_neox': 1025, 'n_words': 244, 'path': 'Machine Learning & Data Science Masterclass - JP/23-Principal-Component-Analysis/01-PCA-with-ScikitLearn.ipynb', 'repo_name': 'ptyadana/probability-and-statistics-for-business-and-data-science', 'size': '3043'}