<jupyter_start><jupyter_text>Learning ObjectivesToday we will be going over just a touch of unsupervised learning, not from a theoretical perspective but from a practical one. We will be asking ourselves how to visualize high dimensional data and will be using PCA in order to do so.<jupyter_code>from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split

iris_data = load_iris()

# we make our test set
X_train, X_test, y_train, y_test = train_test_split(iris_data['data'], iris_data['target'], test_size=0.2, random_state=1)

# and we make our validation set
X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.2, random_state=1)
import numpy as np
import pandas as pd

full_data = np.append(X_train, y_train.reshape(-1, 1), axis=-1)

full_df = pd.DataFrame(data=full_data, columns=np.append(iris_data['feature_names'], 'target'))<jupyter_output><empty_output><jupyter_text>In a supervised learning problem we are interested in learning what a specific of our feature will be on new data points. In unsupervised learning we are interested in emergent facts about our data. Today we will ask ourselves one question. That being said there are many different techniques and this is only the smallest part of unsupervised learning. Here is the question:1. Is there an easy way to visualize our dataset when we have multiple quantitative dims  PCAThe way that we previously visualized the quantitative variables in the billionaires dataset was using the pairplot, we can look at it bellow:<jupyter_code>%matplotlib inline

import seaborn as sns

sns.pairplot(full_df, vars=iris_data['feature_names'])<jupyter_output><empty_output><jupyter_text>In such a way we could see all the quantitative variables in one plot. But this is somewhat hard to see how the flowers clump up, or distinctive features between them all. Or if we wanted to plot a qualitative variable on top and see if it affects the various quanities, it is often hard, eg.:<jupyter_code>sns.pairplot(full_df, hue='target', vars=iris_data['feature_names'])<jupyter_output><empty_output><jupyter_text>For me it is hard to see global patterns, though local patterns stand out. Now the real problem is when the dim gets larger than 5, it is nearly impossible to separate them out. This is where dimensionality reduction techniques like PCA come into play.PCA will reduce the dimension of your dataset to whatever you want and will tell you the tradeoff associated. Let's use PCA below and interpret the results:<jupyter_code>from sklearn.decomposition import PCA
from sklearn.preprocessing import scale

# we choose two components so that we can visualize this in 2D
pca = PCA(2)

# normalize the data before we transform it. This tends to be very important for most ML applications
X = scale(X_train)
transformed_data = pca.fit_transform(X)

transformed_data.shape
print iris_data['feature_names']
print pca.components_
print pca.explained_variance_ratio_<jupyter_output>['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']
[[ 0.52928386 -0.2385349   0.58415298  0.56720807]
 [ 0.33683735  0.9392027   0.01537152  0.06482759]]
[ 0.72087061  0.2347434 ]<jupyter_text>PCA attempts to explain the variance in your data. If your numeric columns are highly related (like age and income) then perhaps you only need one number to represent them both. I printed out two things, the components in PCA and the explained variance. The components represent linear combinations of the original columns.The second thing I printed out is the explained variance ratio. The higher this is the more that the columns returned can explain the full data. In our case our two columns explain 95% of the var in our data.Regardless we are already here, so let's plot the result:<jupyter_code>sns.jointplot(transformed_data[:, 0], transformed_data[:, 1])<jupyter_output><empty_output><jupyter_text>Remember the dimensions no longer represent any concrete features, but rather an aggregation of a couple of features. We can see if these dimensions correspond to specific qualitative features as well:<jupyter_code>data = pd.DataFrame(transformed_data).join(full_df[['target']])
sns.pairplot(data, 'target', vars=[0,1])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06577715355805243, 'fraction_numerical': 0.024812734082397005, 'mean_word_length': 5.771790808240888, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '394508', 'license': 'MIT', 'n_tokens_mistral': 1290, 'n_tokens_neox': 1202, 'n_words': 539, 'path': 'practicals/16.UnsupervisedLearning.ipynb', 'repo_name': 'marxcerqueira/ds-foundations', 'size': '4272'}