<jupyter_start><jupyter_text>"What is Principle Component Analysis (PCA)?"- toc:true- branch: master- badges: true- comments: true- author: <NAME>- categories: [ML, Dimension_Reduction] PCA is a dimension reduction technique widely used for visualising larger dimension data by converting them to two or three dimensions. It is unsupervised statistical technique.It finds the correlation between the features and converts it into uncorrelated data by transforming them into some other dimesion space whose dimentionality is less than the original data's dimentionality.Goal of PCA- Represent data in new dimention such that -->                                1. the data has high variance along these dimetions                                2. the dimentions are lineary independent (i.e. uncorrelated)                                3. dimentions are orthogonal                                Let's try to understand further with a 2-D data.<jupyter_code>#hide
import numpy as np
import random
import matplotlib.pyplot as plt
#hide
random.seed(random)
#lets create artificial 2-D data
x1 = np.linspace(start = -10, stop = +10, num = 11)
x2 = 2*x1 + x1*random.uniform(a = -1, b = +1)
plt.scatter(x1, x2)
plt.title("Dataset")
plt.xlabel('x1')
plt.ylabel('x2')
plt.grid()
plt.show()<jupyter_output><empty_output><jupyter_text>Now we have our artificial data set X where 1st column is for x1 and 2nd column for x2. X has 11 examples (no. of data points) and 2 features x1 and x2. Clearly $x_2 \sim 3*x_1$ So x1 and x2 are highly correlated.Lets consider the direction $u$ which coincides with $x_2 \sim 3*x_1$ and we project all the data point on this line.<jupyter_code>plt.scatter(x1, x2, label = 'original data points')
plt.scatter(x1, 3*x1, label = 'projected data points', marker = 'x', color = 'r')
plt.plot(x1, 3*x1, label = 'u direction')
plt.xlabel('x1')
plt.ylabel('x2')
plt.grid()
plt.legend()
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09469891919711786, 'fraction_numerical': 0.025218733916623777, 'mean_word_length': 3.6066350710900474, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '315972', 'license': 'Apache-2.0', 'n_tokens_mistral': 636, 'n_tokens_neox': 588, 'n_words': 237, 'path': '_notebooks/2020-04-23-tutorial-blog-PCA.ipynb', 'repo_name': 'gen1ze/gen1ze_ml_blog', 'size': '1953'}