<jupyter_start><jupyter_text>k-MEANS CLUSTERING**File:** kMeans.ipynb**Course:** Data Science Foundations: Data Mining in Python  IMPORT LIBRARIES<jupyter_code>import pandas as pd                               # For dataframes
import matplotlib.pyplot as plt                   # For plotting data
import seaborn as sns                             # For plotting data
from sklearn.cluster import KMeans                # For k-Means
from sklearn.model_selection import GridSearchCV  # For grid search
from sklearn.metrics import silhouette_score      # For metrics and scores
from sklearn.preprocessing import StandardScaler  # For standardizing data<jupyter_output><empty_output><jupyter_text>LOAD AND PREPARE DATARead the `penguins.csv` file from the `data` directory in variable `df`. Keep all features in variable `df` and store the class variable in `y`. Also, standardize the data so each variable has a mean of zero and unit variance (i.e., variance and standard deviation of one).<jupyter_code># Reads the .csv file into variable df
df = pd.read_csv('data/penguins.csv')

# Separates the class variable in y
y = df.y

# Removes the y column from df
df = df.drop('y', axis=1)

# Standardizes df
df = pd.DataFrame(
    StandardScaler().fit_transform(df),
    columns=df.columns)

# Displays the first 5 rows of df
df.head()<jupyter_output><empty_output><jupyter_text>RUNNING k-MEANS k-Means: Train the ModelWe'll set up a `KMeans` object with the following parameters:- `n_clusters`: Total number of clusters to make.- `random_state`: Set to one to reproduce these results.- `init`: How to initialize the k-means centers; we'll use `k-means++`.- `n_init`: Number of times k-means would be run; the model returned would have the minimum value of `inertia`.A few attributes of the `KMeans` object, which are also used in this demo are:- `cluster_centers_`: Stores the discovered cluster centers.- `labels_`: Label of each instance.- `inertia`: Sum of square of distances of each instance from its corresponding center.- `n_iter`: Number of iterations run to find the centers.<jupyter_code># Sets up the kMeans object
km = KMeans(
    n_clusters=3,
    random_state=1,
    init='k-means++',
    n_init=10)

# Fits the model to the data
km.fit(df)

# Displays the parameters of the fitted model
km.get_params()<jupyter_output><empty_output><jupyter_text>k-Means: Visualize the ClustersThe code below creates a scatterplot of the first two features. Each point is colored according to its actual label. For comparison, each instance is drawn with a marker according to the label found by the clustering algorithm.<jupyter_code># Creates a scatter plot
sns.scatterplot(
    x='bill_length_mm', 
    y='bill_depth_mm',
    data=df, 
    hue=y,
    style=km.labels_,
    palette=["orange", "green", "blue"])

# Adds cluster centers to the same plot
plt.scatter(
    km.cluster_centers_[:,0],
    km.cluster_centers_[:,1],
    marker='x',
    s=200,
    c='red')<jupyter_output><empty_output><jupyter_text>k-MEANS: OPTIMIZE VIA SILHOUETTE SCORESThe main challenge in k-means is to find the optimal number of clusters. We can set up a `GridSearchCV` object to search for the optimal parameters. For k-Mmeans, we require a custom scorer that computes the silhouette value for different number of clusters specified by `n_clusters`. The custom scorer is called `s2()` in the code below, where it uses `silhouette_score()` from the `sklearn.metrics` library to compute a score for an instance `X`. A silhouette score is a value in [-1,+1]. It is a means for comparing how similar an instance is to its corresponding cluster compared to its similarity with other clusters. Formally, it takes into account `cohesion` and `separation` to compute a silhouette value. A +1 or close to this score value indicates better clusters.<jupyter_code># Sets up the custom scorer
def s2(estimator,X):
    return silhouette_score(X, estimator.predict(X))

# List of values for the parameter `n_clusters`
param = range(2,10)

# KMeans object
km = KMeans(random_state=0, init='k-means++')

# Sets up GridSearchCV object and stores in grid variable
grid = GridSearchCV(
    km,
    {'n_clusters': param},
    scoring=s2,
    cv=2)

# Fits the grid object to data
grid.fit(df)

# Accesses the optimum model
best_km = grid.best_estimator_

# Displays the optimum model
best_km.get_params()<jupyter_output><empty_output><jupyter_text>Plot of Scores for Different Number of ClustersThe `grid` object has an attribute `cv_results_` through which the scores for different `n_clusters` can be accessed.<jupyter_code># Plot mean_test_scores vs. n_clusters
plt.plot(
    param,
    grid.cv_results_['mean_test_score'])

# Draw a vertical line, where the best model is
plt.axvline(
    x=best_km.n_clusters, 
    color='red',
    ls='--')

# Adds labels to the plot
plt.xlabel('Total Centers')
plt.ylabel('Silhouette Score')<jupyter_output><empty_output><jupyter_text>Visualize the Best ModelCode below creates a visualization of the clusters stored in the optimum model `best_km`.<jupyter_code># Creates a scatter plot
sns.scatterplot(
    x='bill_length_mm', 
    y='bill_depth_mm',
    data=df, 
    hue=y,
    style=best_km.labels_,
    palette=['orange', 'green', 'blue'])

# Adds cluster centers to the same plot
plt.scatter(
    best_km.cluster_centers_[:, 0],
    best_km.cluster_centers_[:, 1],
    marker='x',
    s=200,
    c='red')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09052437902483901, 'fraction_numerical': 0.004967801287948482, 'mean_word_length': 4.3241919686581785, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '356192', 'license': 'CECILL-B', 'n_tokens_mistral': 1756, 'n_tokens_neox': 1676, 'n_words': 626, 'path': 'BC_kMeans.ipynb', 'repo_name': 'JonWeber0328/Data_Mining_In_Python_LinkedIn_Learning', 'size': '5435'}