<jupyter_start><jupyter_text>Understanding PCA behaviour June 2, 2019 <NAME>This notebook explores how the Principal Component Analysis works by making use of synthetic data.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
def random_df(mx=0, sx=1, my=0, sy=1, n=1000):
    '''Generates a random 2D normal dataframe'''
    x = np.random.normal(mx, sx, 1000)
    y = np.random.normal(mx, sy, 1000)
    
    return pd.DataFrame({'x': x, 'y':y})

def plot_xy(x,y):
    '''Makes a scatter plot'''
    plt.scatter(x, y)
    M = max(x.max(), y.max())
    m = min(x.min(), y.min())
    plt.xlim(m, M)
    plt.ylim(m, M)
    #plt.gca().set_aspect('equal', adjustable='box')
    plt.show()
    
def pca_components(df=None, mx=0, sx=1, my=0, sy=1, n=1000, x='x', y='y'):
    '''Generates a dataset if needed, performs PCA and prints the components and variance ratio'''
    if df is None:
        df = random_df(mx, sx, my, sy, n)
    plot_xy(df[x], df[y])
    pca = PCA()
    pca.fit(df)
    print('Components:')
    print(pca.components_)
    print('Variance ratio:')
    print(pca.explained_variance_ratio_)<jupyter_output><empty_output><jupyter_text>Two normal uncorrelated variablesThe variance ratio of the components is almost the same, meaning that one needs to have information for each variable to define the dataset. The components are composed by a biased weight distribution.<jupyter_code>pca_components()<jupyter_output><empty_output><jupyter_text>One normal variable and one constantAll the variance is explained by the normally distributed variable.<jupyter_code>pca_components(mx=0, sx=10, my=0, sy=0)<jupyter_output><empty_output><jupyter_text>Two normal uncorrelated variables with different variancesThe variance ratio of the components leans to the higher variance distribution.<jupyter_code>pca_components(mx=0, sx=10, my=0, sy=1)
pca_components(mx=0, sx=1, my=0, sy=10)<jupyter_output><empty_output><jupyter_text>One normal and one chi squared distributionThe variance ratio of the components is almost the same, meaning that one needs to have information for each variable to define the dataset. The components are composed by a very biased weight distribution, meaning that each variable has a very different behaviour than the other.<jupyter_code>pca_components(pd.DataFrame({'x': np.random.chisquare(1,1000), 'y':np.random.normal(0, 1, 1000)}))<jupyter_output><empty_output><jupyter_text>Two correlated chisquared variablesThe variance ratio of the components leans highly to one of the variables, meaning that if we have information for that variable then we know how the dataset varies entirely. Nevertheless, the components of the first PC have about an equal contribution of each variable, meaning that we need to have the levels of both to get all the information in.<jupyter_code>x = np.random.chisquare(50,1000)
pca_components(pd.DataFrame({'x': x, 'y': x * 1.12}))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10491146007350484, 'fraction_numerical': 0.021717340461075842, 'mean_word_length': 5.3298097251585626, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '67055', 'license': 'MIT', 'n_tokens_mistral': 994, 'n_tokens_neox': 938, 'n_words': 324, 'path': 'Dimensionality Reduction/PCA test/PCA behaviour.ipynb', 'repo_name': 'luisds95/RLexercises', 'size': '3000'}