<jupyter_start><jupyter_text>The $k$-Means Algorithm====================Again, we start by generating some artificial data:<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
#from IPython.display import HTML
# This line tells the notebook to show plots inside of the notebook
%matplotlib inline
plt.jet() # set the color map. When your colors are lost, re-run this.
import sklearn.datasets as datasets
X, Y = datasets.make_blobs(centers=4, cluster_std=0.5, random_state=0)<jupyter_output><empty_output><jupyter_text>As always, we first *plot* the data to get a feeling of what we're dealing with:<jupyter_code>plt.scatter(X[:,0], X[:,1]);<jupyter_output><empty_output><jupyter_text>The data looks like it may contain four different "types" of data point. In fact, this is how it was created above.We can plot this information as well, using color:<jupyter_code>plt.scatter(X[:,0], X[:,1], c=Y);<jupyter_output><empty_output><jupyter_text>Normally, you do not know the information in `Y`, however.You could try to recover it from the data alone.This is what the kMeans algorithm does.<jupyter_code>from sklearn.cluster import KMeans
kmeans = KMeans(4, random_state=8)
Y_hat = kmeans.fit(X).labels_<jupyter_output><empty_output><jupyter_text>Now the label assignments should be quite similar to `Y`, up to a different ordering of the colors:<jupyter_code>plt.scatter(X[:,0], X[:,1], c=Y_hat);<jupyter_output><empty_output><jupyter_text>Often, you're not so much interested in the assignments to the means. You'll want to have a closer look at the means $\mu$.The means in $\mu$ can be seen as *representatives* of their respective cluster.<jupyter_code>plt.scatter(X[:,0], X[:,1], c=Y_hat, alpha=0.4)
mu = kmeans.cluster_centers_
plt.scatter(mu[:,0], mu[:,1], s=100, c=np.unique(Y_hat))
print mu<jupyter_output>[[-1.47935679  3.11716896]
 [-1.26811733  7.76378266]
 [ 1.99186903  0.96561071]
 [ 0.92578447  4.32475792]]<jupyter_text>$k$-Means on Images In this final example, we use the $k$-Means algorithm on the classical MNIST dataset.The MNIST dataset contains images of hand-written digits. Let's first fetch the dataset from the internet (which may take a while, note the asterisk [*]):<jupyter_code>from sklearn.datasets import fetch_mldata
from sklearn.cluster import KMeans
from sklearn.utils import shuffle
X_digits, _,_, Y_digits = fetch_mldata("MNIST Original").values() # fetch dataset from internet
X_digits, Y_digits = shuffle(X_digits,Y_digits) # shuffle dataset (which is ordered!)
X_digits = X_digits[-5000:]       # take only the last instances, to shorten runtime of KMeans<jupyter_output><empty_output><jupyter_text>Let's have a look at some of the instances in the dataset we just loaded:<jupyter_code>plt.rc("image", cmap="binary") # use black/white palette for plotting
for i in xrange(10):
    plt.subplot(2,5,i+1)
    plt.imshow(X_digits[i].reshape(28,28))
    plt.xticks(())
    plt.yticks(())
plt.tight_layout()<jupyter_output><empty_output><jupyter_text>**Warning**: This takes quite a few seconds, so be patient until the asterisk [*] disappears!<jupyter_code>kmeans = KMeans(20)
mu_digits = kmeans.fit(X_digits).cluster_centers_<jupyter_output><empty_output><jupyter_text>Let's have a closer look at the means. Even though there are 10 digits, some of them are over/under-represented. Do you understand why?<jupyter_code>plt.figure(figsize=(16,6))
for i in xrange(2*(mu_digits.shape[0]/2)): # loop over all means
    plt.subplot(2,mu_digits.shape[0]/2,i+1)
    plt.imshow(mu_digits[i].reshape(28,28))
    plt.xticks(())
    plt.yticks(())
plt.tight_layout()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1231404958677686, 'fraction_numerical': 0.03415977961432507, 'mean_word_length': 6.175889328063241, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '4554', 'license': 'MIT', 'n_tokens_mistral': 1349, 'n_tokens_neox': 1256, 'n_words': 365, 'path': '.ipynb_checkpoints/5 - KMeans-checkpoint.ipynb', 'repo_name': 'guptesanket/Python-for-Neuroscientists', 'size': '3630'}