<jupyter_start><jupyter_text>K-Means Clustering in Python   <NAME>  Overview  1. [What is K-Means Clustering?](section1)2. [Key Terms](section2) 3. [Pros and Cons of K-Means Clustering](section3)4. [When to use K-Means Clustering](section4)5. [Key Parameters](section5) 6. [Walkthrough: Building a k means clustering model](section6)7. [Additional Reading](section7) 8. [Conclusion](section8)9. [Sources](section9)   What is K Means Clustering?  K means clustering is a clustering algorithm. Clustering algorithms are unsupervised learning methods used to find non overlapping groups within a set of data. Typically, clustering (and unsupervised methods in general) are used in the exploratory data analysis phase of a machine learning project to better understand the underlying structure of our data. Clustering is especially common in medical and marketing settings where it can be used to more clearly understand patient groups and customer subgroups, respectively. K means clustering is a straightforward and effective method of splitting data into k non overlapping groups. To perform k means clustering we first specify the number of clusters we want in our data, then the clustering algorithm assigns every observation to the group to which it most closely belongs. More specifically, a number (1 through k) is randomly assigned to each observation in the data. Next, a centroid is computed for each for each of the k clusters, where the kth cluster centroid is a point obtained by taking the means of all the features within the cluster. Lastly, the algorithm reassigns all the points to the cluster based on the centroid to which they are the closest. K means clustering gives the local optimum, so it is important to run the algorithm multiple times to ensure the best solution arises.   Visualizations of K Means Clustering        Key Terms  1. **Clustering**: Unsupervised learning technique to find non-overlapping subgroups within a dataset. Commonly used to find customer or patient groups, analomies, or to get a general data visualization.2. **K**: The number of clusters, selected by the user. 3. **Centroid**: The average center of mass within a cluster. Used to help determine what observation in a dataset belongs in which cluster.    Pros and Cons of K Means Clustering   Pros  1. **Simple**: The math behind k means clustering is not overly complex and the results are easy to interpret, visualize, and explain. 2. **Efficient**: K means clustering runs effectively on large or datasets. 3. **Flexible**: K means clustering can be applied to a variety of problems and has adjustable parameters. 4. **Fast**: One of the fastest clustering algorithms since its complexity is only O(knT), where k,n and T are the number of clusters, samples and iterations, respectively. Cons  1. **Non Optimal Cluster Numbers**: Since the user has to specify k, the clustering algorithm cannot find and create the optimal number of clusters. 2. **Finds Local Minima**: K means clustering finds the local optimum, not the globally best solution. To combat this, users need to run the algorithm multiple times to ensure they are getting good results. 3. **Lack of Validation**: this is a problem of unsupervised learning generally, but there is no agreed upon way in which to verify the accuracy of unsupervised learning models because they lack a target variable to use as an accuracy measure. 4. **Does Poorly on Non-Globular Data**: K means tends to make circular/globular clusters. If one's data is a strange shape, like a swirl, k means fail miserably.    When to use K Means Clustering  K means clustering works well when one is interested in understanding the shape of their data, the subgroups within their data, or if there is interest in identifying outlying data points. k-means clustering is commonly used in marketing, research studies, realty (predicting house prices), fraud detection, document classification, medicine, and imagery processing.     Key Parameters   **n_clusters**: i.e. K, the number of clusters**init**: This is how the clusters are initially created. One can specify the initial centroid points with an array, random starting points, or accept the defaults of a package like scikit learn. **n_init**: The number of times to run the the k means model. Since k means clustering can get stuck giving a local optimum instead of a global, it is important to run the model multiple times    Walkthrough: Building a K Means Clustering Model  In this walkthroug we will stepping throug a k means clustering model to visualize the [digits dataset](https://scikit-learn.org/stable/auto_examples/datasets/plot_digits_last_image.html). The digits dataset contains handwritten numbers (0-9). Ultimately, we would expect to see 9 different groups form given the differences in each number. We will perform the following steps: 1. import the necessary packages 2. load the dataset 3. scale the data (this is done because k means uses euclidean distance, so everything needs to be scaled relative to each other)4. examine and visualize our data 5. create 3 clustering models with different initializations6. compare the accuracy of the 3 models 7. create a visualization for the most accurate model<jupyter_code># import necessary packages 

from time import time 
import matplotlib.pyplot as plt
import numpy as np 
from sklearn import metrics 
from sklearn.cluster import KMeans
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.preprocessing import scale 
# load our dataset 

digits_dataset=load_digits()
# scale the data 

data=scale(digits_data.data)
# examine the data 

print('dataset shape:',data.shape)
print('dataset type:',data.dtype)
print('dataset head:',data[0:3]) #these are arrays of pixel coordinates used to print the digits dataset (these ones are scaled)
# lets check out some examples of the digits we might encounter

for i in range(1,10):
    plt.figure(1, figsize=(3, 3))
    plt.imshow(digits_dataset.images[i], cmap=plt.cm.gray_r, interpolation='nearest') #change the '0' in this line to see different numbers
    plt.show()
#specify our parameters for the clustering model 

n_samples, n_features = data.shape
n_digits = len(np.unique(digits_dataset.target))
labels = digits_dataset.target
sample_size = 300

print("n_digits: %d, \t n_samples: %d, \t n_features: %d"
      % (n_digits, n_samples, n_features))

# create a function to run our k means classifier and record its metadata

def bench_k_means(estimator, name, data):
    t0 = time()
    estimator.fit(data)
    print('%-9s\t%.2fs\t%i\t%.3f\t%.3f\t%.3f\t%.3f\t%.3f\t%.3f'
          % (name, (time() - t0), estimator.inertia_,
             metrics.homogeneity_score(labels, estimator.labels_),
             metrics.completeness_score(labels, estimator.labels_),
             metrics.v_measure_score(labels, estimator.labels_),
             metrics.adjusted_rand_score(labels, estimator.labels_),
             metrics.adjusted_mutual_info_score(labels,  estimator.labels_),
             metrics.silhouette_score(data, estimator.labels_,
                                      metric='euclidean',
                                      sample_size=sample_size)))

# run the model with different types of centroid initializations
# first we initialize with k means ++, then random, then using the principal components of the digit masses 
# for more info on principal components check out the reference guide on PCA

print('init\t\ttime\tinertia\thomo\tcompl\tv-meas\tARI\tAMI\tsilhouette')

bench_k_means(KMeans(init='k-means++', n_clusters=n_digits, n_init=10),
              name="k-means++", data=data)

bench_k_means(KMeans(init='random', n_clusters=n_digits, n_init=10),
              name="random", data=data)

# in this case the seeding of the centers is deterministic, we run the algorithm only once with n_init=1

pca = PCA(n_components=n_digits).fit(data)
bench_k_means(KMeans(init=pca.components_, n_clusters=n_digits, n_init=1),
              name="PCA-based",
              data=data)


print('_')

# let's create a visualization of our PCA model components 

reduced_data = PCA(n_components=2).fit_transform(data)
kmeans = KMeans(init='k-means++', n_clusters=n_digits, n_init=10)
kmeans.fit(reduced_data)

# Step size of the mesh. Decrease h to increase the quality of the VQ.
h = .02     # point in the mesh [x_min, x_max]x[y_min, y_max].


# Plot the decision boundary. For that, we will assign a color to each

x_min, x_max = reduced_data[:, 0].min() - 1, reduced_data[:, 0].max() + 1
y_min, y_max = reduced_data[:, 1].min() - 1, reduced_data[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))


# Obtain labels for each point in mesh. Use last trained model.

Z = kmeans.predict(np.c_[xx.ravel(), yy.ravel()])


# Put the result into a color plot

Z = Z.reshape(xx.shape)
plt.figure(figsize=(12,9))
plt.clf()
plt.imshow(Z, interpolation='nearest',
           extent=(xx.min(), xx.max(), yy.min(), yy.max()),
           cmap=plt.cm.Paired,
           aspect='auto', origin='lower')

plt.plot(reduced_data[:, 0], reduced_data[:, 1], 'k.', markersize=2)

# Plot the centroids as a white X

centroids = kmeans.cluster_centers_
plt.scatter(centroids[:, 0], centroids[:, 1],
            marker='x', s=169, linewidths=3,
            color='w', zorder=10)

# define the plot title and bounds

plt.title('K-means clustering on digits dataset (PCA-reduced data)\n'
          'Centroids are marked with white cross')
plt.xlim(x_min, x_max)
plt.ylim(y_min, y_max)
plt.xticks(())
plt.yticks(())
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08298712089738264, 'fraction_numerical': 0.01007478188616535, 'mean_word_length': 4.258874931731294, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '533035', 'license': 'MIT', 'n_tokens_mistral': 2843, 'n_tokens_neox': 2631, 'n_words': 1194, 'path': 'k_means_clustering/k_means_clustering.ipynb', 'repo_name': 'kcmckee/reference_guides', 'size': '9635'}