<jupyter_start><jupyter_text>Exercise 10.10    In this problem, you will generate simulated data, and then perform PCA and $K$-means clustering on the data.                        Generate a simulated data set with 20 observations in each of three classes (i.e. 60 observations total), and 50 variables.                                         Hint: There are a number of functions in R that you can use to generate data. One example is the rnorm() function; runif() is another option. Be sure to add a mean shift to the observations in each class so that there are three distinct classes.                            Perform PCA on the 60 observations and plot the first two principal component score vectors. Use a different color to indicate the observations in each of the three classes. If the three classes appear separated in this plot, then continue on to part 3. If not, then return to part 1 and modify the simulation so that there is greater separation between the three classes. Do not continue to part 3 until the three classes show at least some separation in the first two principal component score vectors.                    Perform $K$-means clustering of the observations with $K = 3$. How well do the clusters that you obtained in $K$-means clustering compare to the true class labels?                                        Hint: You can use the table() function in R to compare the true class labels to the class labels obtained by clustering. Be careful how you interpret the results: $K$-means clustering will arbitrarily number the clusters, so you cannot simply check whether the true class labels and clustering labels are the same.                            Perform $K$-means clustering with $K = 2$. Describe your results.        Now perform $K$-means clustering with $K = 4$, and describe your results.        Now perform $K$-means clustering with $K = 3$ on the first two principal component score vectors, rather than on the raw data. That is, perform $K$-means clustering on the 60 $\times$ 2 matrix of which the first column is the first principal component score vector, and the second column is the second principal component score vector. Comment on the results.        Using the scale() function, perform $K$-means clustering with $K = 3$ on the data after scaling each variable to have standard deviation one. How do these results compare to those obtained in 2? Explain.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt


# https://stackoverflow.com/questions/34398054/ipython-notebook-cell-multiple-outputs
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all" 

from sklearn.datasets import make_blobs
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.pipeline import Pipeline
from sklearn.cluster import KMeans<jupyter_output><empty_output><jupyter_text>Exercise 10.10.1            Generate a simulated data set with 20 observations in each of three classes (i.e. 60 observations total), and 50 variables.                             Hint: There are a number of functions in R that you can use to generate data. One example is the rnorm() function; runif() is another option. Be sure to add a mean shift to the observations in each class so that there are three distinct classes.<jupyter_code>n = 60
n_per_class = 20
p = 50
classes = ['No', 'Maybe', 'Yes']

x, y = make_blobs(
    n_samples=n,  # If int, it is the total number of points equally divided among clusters.
    n_features=50,
    centers = 3,
    cluster_std=[3, 1.5, 2],
    shuffle=True,
    random_state=42
)

df_y = pd.DataFrame({'y': y})
df_y.loc[(df_y['y'] == 0, 'y')] = 'No'
df_y.loc[(df_y['y'] == 1, 'y')] = 'Maybe'
df_y.loc[(df_y['y'] == 2, 'y')] = 'Yes'
df_x = pd.DataFrame()
for i, (class_, column) in enumerate(zip(classes, x.T)):
    df_x[class_] = column<jupyter_output><empty_output><jupyter_text>Exercise 10.10.2    Perform PCA on the 60 observations and plot the first two principal component score vectors. Use a different color to indicate the observations in each of the three classes. If the three classes appear separated in this plot, then continue on to part 3. If not, then return to part 1 and modify the simulation so that there is greater separation between the three classes. Do not continue to part 3 until the three classes show at least some separation in the first two principal component score vectors.<jupyter_code>pipeline = Pipeline([
    ('scaling', StandardScaler()), 
    ('pca', PCA(n_components=2))
])
principal_components = pipeline.fit_transform(df_x)
pipeline['pca'].explained_variance_ratio_
colors = ['r', 'g', 'b']
fig, ax = plt.subplots(1, 1, constrained_layout=True, figsize=(8, 6))
for class_, color in zip(classes, colors):
    mask = df_y['y'] == class_
    _ = ax.scatter(
        principal_components[:, 0][mask],
        principal_components[:, 1][mask],
        c=color,
        label=class_
    )

_ = ax.set_xlabel('principal component 1')
_ = ax.set_ylabel('principal component 2')
_ = ax.legend()<jupyter_output><empty_output><jupyter_text>Exercise 10.10.3            Perform $K$-means clustering of the observations with $K = 3$. How well do the clusters that you obtained in $K$-means clustering compare to the true class labels?                            Hint: You can use the table() function in R to compare the true class labels to the class labels obtained by clustering. Be careful how you interpret the results: $K$-means clustering will arbitrarily number the clusters, so you cannot simply check whether the true class labels and clustering labels are the same.<jupyter_code>k = 3
kmeans = KMeans(n_clusters=k, n_jobs=-1, random_state=42)
y_pred = kmeans.fit_predict(df_x)
df_y_pred = pd.DataFrame({'y': y_pred})
classes = np.unique(y_pred)

fig, ax = plt.subplots(1, 1, constrained_layout=True, figsize=(8, 6))
for i, class_ in enumerate(classes):
    mask = df_y_pred['y'] == class_
    _ = ax.scatter(
        principal_components[:, 0][mask],
        principal_components[:, 1][mask],
        label=f'class {i + 1}'
    )

_ = ax.set_xlabel('principal component 1')
_ = ax.set_ylabel('principal component 2')
_ = ax.legend()<jupyter_output><empty_output><jupyter_text>The $K$-means clustering with $K = 3$ assigns all the instances to the appropriate cluster.Exercise 10.10.4    Perform $K$-means clustering with $K = 2$. Describe your results.<jupyter_code>k = 2
kmeans = KMeans(n_clusters=k, n_jobs=-1, random_state=42)
y_pred = kmeans.fit_predict(df_x)
df_y_pred = pd.DataFrame({'y': y_pred})
classes = np.unique(y_pred)

fig, ax = plt.subplots(1, 1, constrained_layout=True, figsize=(8, 6))
for i, class_ in enumerate(classes):
    mask = df_y_pred['y'] == class_
    _ = ax.scatter(
        principal_components[:, 0][mask],
        principal_components[:, 1][mask],
        label=f'class {i + 1}'
    )

_ = ax.set_xlabel('principal component 1')
_ = ax.set_ylabel('principal component 2')
_ = ax.legend()<jupyter_output><empty_output><jupyter_text>Exercise 10.10.5    Now perform $K$-means clustering with $K = 4$, and describe your results.<jupyter_code>k = 4
kmeans = KMeans(n_clusters=k, n_jobs=-1, random_state=42)
y_pred = kmeans.fit_predict(df_x)
df_y_pred = pd.DataFrame({'y': y_pred})
classes = np.unique(y_pred)

fig, ax = plt.subplots(1, 1, constrained_layout=True, figsize=(8, 6))
for i, class_ in enumerate(classes):
    mask = df_y_pred['y'] == class_
    _ = ax.scatter(
        principal_components[:, 0][mask],
        principal_components[:, 1][mask],
        label=f'class {i + 1}'
    )

_ = ax.set_xlabel('principal component 1')
_ = ax.set_ylabel('principal component 2')
_ = ax.legend()<jupyter_output><empty_output><jupyter_text>Exercise 10.10.6    Now perform $K$-means clustering with $K = 3$ on the first two principal component score vectors, rather than on the raw data. That is, perform $K$-means clustering on the 60 $\times$ 2 matrix of which the first column is the first principal component score vector, and the second column is the second principal component score vector. Comment on the results.<jupyter_code>k = 3
kmeans = KMeans(n_clusters=k, n_jobs=-1, random_state=42)
y_pred = kmeans.fit_predict(principal_components)
df_y_pred = pd.DataFrame({'y': y_pred})
classes = np.unique(y_pred)

fig, ax = plt.subplots(1, 1, constrained_layout=True, figsize=(8, 6))
for i, class_ in enumerate(classes):
    mask = df_y_pred['y'] == class_
    _ = ax.scatter(
        principal_components[:, 0][mask],
        principal_components[:, 1][mask],
        label=f'class {i + 1}'
    )

_ = ax.set_xlabel('principal component 1')
_ = ax.set_ylabel('principal component 2')
_ = ax.legend()<jupyter_output><empty_output><jupyter_text>Exercise 10.10.7    Using the scale() function, perform $K$-means clustering with $K = 3$ on the data after scaling each variable to have standard deviation one. How do these results compare to those obtained in 2? Explain.<jupyter_code>k = 3
pipeline = Pipeline([
    ('scaling', StandardScaler()), 
    ('kmeans', KMeans(n_clusters=k, n_jobs=-1, random_state=42))
])
y_pred = pipeline.fit_predict(principal_components)
df_y_pred = pd.DataFrame({'y': y_pred})
classes = np.unique(y_pred)

fig, ax = plt.subplots(1, 1, constrained_layout=True, figsize=(8, 6))
for i, class_ in enumerate(classes):
    mask = df_y_pred['y'] == class_
    _ = ax.scatter(
        principal_components[:, 0][mask],
        principal_components[:, 1][mask],
        label=f'class {i + 1}'
    )

_ = ax.set_xlabel('principal component 1')
_ = ax.set_ylabel('principal component 2')
_ = ax.legend()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10173338836153528, 'fraction_numerical': 0.018572018159306643, 'mean_word_length': 3.9504596527068436, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '608992', 'license': 'MIT', 'n_tokens_mistral': 3114, 'n_tokens_neox': 2890, 'n_words': 1084, 'path': 'Chapter10/Exercise10/10_10.ipynb', 'repo_name': 'HerrBroekman/Applied-solution-of-An-Introduction-to-Statistical-Learning-written-in-Python', 'size': '9692'}