<jupyter_start><jupyter_text>K-means Clustering Clustering is the process of dividing the set of objects into groups (clusters) in such way that objects in the same group are more similar to each other then to the ojects from other groups. ![clusters.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA+gAAAH0CAYAAACuKActAAAEDmlDQ1BrQ0dDb2xvclNwYWNlR2VuZXJpY1JHQgAAOI2NVV1oHFUUPpu5syskzoPUpqaSDv41lLRsUtGE2uj+ZbNt3CyTbLRBkMns3Z1pJjPj/KRpKT4UQRDBqOCT4P9bwSchaqvtiy2itFCiBIMo+ND6R6HSFwnruTOzu5O4a73L3PnmnO9+595z7t4LkLgsW5beJQIsGq4t5dPis8fmxMQ6dMF90A190C0rjpUqlSYBG+PCv9rt7yDG3tf2t/f/Z+uuUEcBiN2F2Kw4yiLiZQD+FcWyXYAEQfvICddi+AnEO2ycIOISw7UAVxieD/Cyz5mRMohfRSwoqoz+xNuIB+cj9loEB3Pw2448NaitKSLLRck2q5pOI9O9g/t/tkXda8Tbg0+PszB9FN8DuPaXKnKW4YcQn1Xk3HSIry5ps8UQ/2W5aQnxIwBdu7yFcgrxPsRjVXu8HOh0qao30cArp9SZZxDfg3h1wTzKxu5E/LUxX5wKdX5SnAzmDx4A4OIqLbB69yMesE1pKojLjVdoNsfyiPi45hZmAn3uLWdpOtfQOaVmikEs7ovj8hFWpz7EV6mel0L9Xy23FMYlPYZenAx0yDB1/<KEY>//<KEY>//<KEY>//<KEY>IAAAQIECBAgQIAAAQJ1BATodVBcIkCAAAECBAgQIECAAAECeQsI0PMWVx4BAgQIECBAgAABAgQIEKgj8H9gKZ8QjBPJJwAAAABJRU5ErkJggg==) Here are several examples:1. Products in the supermarket are organised into sections: bakery, dairy, fruits, vegetables, frozen, meat, cleaning supplies, etc.2. Movies on Netflix are grouped into categories: Action, Drama, Historical, Horror, etc.3. Songs on Spotify are grouped by musical styles: Rock'n'Roll, Pop, Jazz, Hip Hop, etc., by mood: Party, Chill, Concentrate, by language: French music, Spanish music, etc.In machine learning, clustering is a problem of unsupervised learning when you are given the unlabeled dataset and asked to partition the data points into several groups, based on a given similarity metric. **Objective:** $$\min_{\mu}\min_{C}\sum_{i=1}^k\sum_{x \in C_i}|x-\mu_i|^2$$ **Step 1.** Fix $\mu$, optimize $C$: $$\min_C\sum_{i=1}^k\sum_{x \in C_i}|x-\mu_i|^2 = \min_C\sum_{i=1}^n|x-\mu_i|^2$$ **Step 2.** Fix $C$, optimize $\mu$: $$\min_{\mu}\sum_{i=1}^k\sum_{x \in C_i}|x-\mu_i|^2$$  Implementation in Python Let's start by loading several Python libraries that will allow us to work with data, visualize the results, etc.<jupyter_code>import pandas as pd # data frame library that allows us to load the data and work with it
import matplotlib.pyplot as plt # data visualization library
import seaborn as sns # another data visualization library built on top of matplotlib. It allows us to have better plots
from sklearn.cluster import KMeans # K-means clustering implemented by scikit-learn library

sns.set_theme(
    style='ticks',
    font_scale=1.3,
    rc={
      'figure.figsize':(11.7,8.27),
      'axes.spines.top': False,
      'axes.spines.right': False
    })<jupyter_output><empty_output><jupyter_text>Step 1. Loading and exploring the data<jupyter_code># Raw CSV file stored in our repository on GitHub
url = 'https://raw.githubusercontent.com/olekscode/arolla-ml/master/data/old-faithful.csv';

# Read the data
data = pd.read_csv(url);

# Display first 5 colums of the dataset
data.head()
data.shape<jupyter_output><empty_output><jupyter_text>Step 2: Visualizing the data<jupyter_code>plt.scatter(x=data['eruptions'], y=data['waiting'])
plt.title('Eruptions of the Old Faithful Geizer')
plt.xlabel("Eruption duration (sec)")
plt.ylabel("Waiting time (min)")<jupyter_output><empty_output><jupyter_text>Step 3. Simple implementation of K-means clustering<jupyter_code><jupyter_output><empty_output><jupyter_text>Step 4. K-means clustering with scikit-learn<jupyter_code>kmeans = KMeans(n_clusters=2)
kmeans.fit(data)
labels = kmeans.labels_
centers = kmeans.cluster_centers_
colors = [sns.color_palette()[i] for i in labels]

plt.title('Eruptions of the Old Faithful Geizer')
plt.xlabel("Eruption duration (sec)")
plt.ylabel("Waiting time (min)")

plt.scatter(x=data['eruptions'], y=data['waiting'], c=colors)

plt.scatter(centers[:, 0], centers[:, 1], c='black', s=200, marker='*')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10980891719745223, 'fraction_numerical': 0.03694267515923567, 'mean_word_length': 7.012244897959183, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '275120', 'license': 'CC0-1.0', 'n_tokens_mistral': 1707, 'n_tokens_neox': 1579, 'n_words': 331, 'path': 'lectures/kmeans/kmeans.ipynb', 'repo_name': 'olekscode/arolla-ml', 'size': '158207'}