<jupyter_start><jupyter_text>K-Means The code written isn't going to be optimized, as it isn't the goal of this notebook. The purpose of this notebook is to present to you an intuitive and easy way to understand how K-means, a simple clustering algorithms, works. Clustering algorithms are used to group points into same clusters. When we have many different attributes, it is extremely difficult to visualize what points are in the same cluster. Clustering is extremely useful, as it gives us an idea of which points are close together.Clustering is used for Unsupervised Learning (if we already have the target values, we already have the clusters).For this example, we will work with a two dimensional cluster. We will extract some points from the iris dataset, and then dive right into the algorithm.<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
from sklearn import datasets
iris = datasets.load_iris()
X = np.array(iris.data[:, 2:])

fig, ax = plt.subplots(figsize = [10,8])
plt.scatter(X[:, 0], X[:, 1], s = 40);<jupyter_output><empty_output><jupyter_text>The Iris dataset is very well known. It has been studied countless of times, and is, alongside the MNIST dataset, the most common test sets. For more information, please readhttps://en.wikipedia.org/wiki/Iris_flower_data_setK-means is an algorithm that consists of three steps:The first step is to randomly K centroids.The second is to identify what points are closest to what centroidThe third is to update the centroids position by taking the average x and y values for that centroid.The second and third step are repeating, and we will go through the math. For now, let's do step 1. We will need to randomly create centroids. Instead of randomly assigning a point in the graph, we will randomly select k points from our data, and set those as our initial centroids.<jupyter_code># Number of clusters
k = 3
# X coordinates of random centroids
C = np.empty((3,2))
for q in range(k):
    random =np.random.randint(X.shape[0])
    C[q,:] = X[random,:]

#Store the initial centroids in a separate array
initC = C.copy()

fig, ax = plt.subplots(figsize = [10,8])
plt.scatter(X[:, 0], X[:, 1], s = 40);
plt.scatter(C[0,0],C[0,1], marker='*', s=200, c='r');
plt.scatter(C[1,0],C[1,1], marker='*', s=200, c='y');
plt.scatter(C[2,0],C[2,1], marker='*', s=200, c='g');<jupyter_output><empty_output><jupyter_text>Now that we have our centroids created, we need to color our other points with the closest centroid's color. For this, we will have to calculate the distance of each point with the centroid.Recall that the distance of two points is given by:$$dist(A,B) = \sqrt{(A_x-B_x)^2+(A_y-B_y)^2} $$<jupyter_code>def distance(X,centroid):
    #Creating Ax
    smallx = X[0]
    smally = X[1]
    dist = []
    for n in range(centroid.shape[0]):
        #Creating Bx
        centX = centroid[n,0]
        centY = centroid[n,1]
        
        #Calculating Distnace
        disty = np.sqrt((smallx-centX)**2 + (smally-centY)**2)
        dist.append(disty)
    #Taking the position of the minumum
    return np.argmin(dist)<jupyter_output><empty_output><jupyter_text>Now that we created our distances, let's color each point with their respective centroid's color.<jupyter_code>colors = ['r', 'y', 'g']
fig, ax = plt.subplots(figsize = [10,8])
#Keeping the closest centroid in an array (we will use this later)
pointcent = []
for n in range(X.shape[0]):
        i = distance(X[n,:], C)
        pointcent.append(i)
        ax.scatter(X[n, 0], X[n, 1], s=40,alpha=0.4, c=colors[i])


plt.scatter(C[0,0],C[0,1], marker='*', s=200, c='r');
plt.scatter(C[1,0],C[1,1], marker='*', s=200, c='y');
plt.scatter(C[2,0],C[2,1], marker='*', s=200, c='g');<jupyter_output><empty_output><jupyter_text>We are almost there! Now we just need to calculate the new centroids. We can use our pointcent array to slice our X array, and use that to calculate the new centroid.<jupyter_code>#For Visualization purposes, we will create oldC and newC to track the change
oldC = C.copy()
for m in range(k):
    indexing = np.array(pointcent)
    C[m,:] = np.mean(X[indexing == m, :], axis = 0)<jupyter_output><empty_output><jupyter_text>Let's plot it.<jupyter_code>colors = ['r', 'y', 'g']
fig, ax = plt.subplots(figsize = [10,8])
#Keeping the closest centroid in an array (we will use this later)
for n in range(X.shape[0]):
        ax.scatter(X[n, 0], X[n, 1], s=40, alpha=0.4, c=colors[pointcent[n]])

plt.scatter(initC[0,0],initC[0,1], marker='*', s=200, c='r');
plt.scatter(initC[1,0],initC[1,1], marker='*', s=200, c='y');
plt.scatter(initC[2,0],initC[2,1], marker='*', s=200, c='g');

#Keeping track of the centroid
plt.plot([oldC[0,0],C[0,0]],[oldC[0,1],C[0,1]],c='k');
plt.plot([oldC[1,0],C[1,0]],[oldC[1,1],C[1,1]],c='k');
plt.plot([oldC[2,0],C[2,0]],[oldC[2,1],C[2,1]],c='k');<jupyter_output><empty_output><jupyter_text>We are now done with our algorithm! We created step1, step2 and step3. We just need to compile everything, adding the amount of loops we want to iterate through.<jupyter_code>#Distance function
def distance(X,centroid):
    #Creating Ax
    smallx = X[0]
    smally = X[1]
    dist = []
    for n in range(centroid.shape[0]):
        #Creating Bx
        centX = centroid[n,0]
        centY = centroid[n,1]
        
        #Calculating Distnace
        disty = np.sqrt((smallx-centX)**2 + (smally-centY)**2)
        dist.append(disty)
    #Taking the position of the minumum
    return np.argmin(dist)


#Step 1: Create K number of centroids
# Number of clusters
k = 3

# X coordinates of random centroids
C = np.empty((3,2))
for q in range(k):
    random =np.random.randint(X.shape[0])
    C[q,:] = X[random,:]

#Store the initial centroids in a separate array
initC = C.copy()

#Start iteration
it = 5
fig, ax = plt.subplots(figsize = [10,8])
for  _ in range(it):
    
    #Step 2: Calculate distance, store minium value in pointcent.
    pointcent = []
    for n in range(X.shape[0]):
        i = distance(X[n,:], C)
        pointcent.append(i)
    
    #Step 3: Update centroid's position
    oldC = C.copy()
    for m in range(k):
        indexing = np.array(pointcent)
        C[m,:] = np.mean(X[indexing == m, :], axis = 0)
    #Optional: Plot the centroid's movements
    for centr in range(k):
        plt.plot([oldC[centr,0],C[centr,0]],[oldC[centr,1],C[centr,1]],c='k');
    
#Plot the end result
for n in range(X.shape[0]):
        ax.scatter(X[n, 0], X[n, 1], s=40, alpha=0.4, c=colors[pointcent[n]])

#Plot initial centroid position
plt.scatter(initC[0,0],initC[0,1], marker='*', s=200, c='r');
plt.scatter(initC[1,0],initC[1,1], marker='*', s=200, c='y');
plt.scatter(initC[2,0],initC[2,1], marker='*', s=200, c='g');<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14871263687481504, 'fraction_numerical': 0.02841077241787511, 'mean_word_length': 4.334648776637727, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '143776', 'license': 'MIT', 'n_tokens_mistral': 2524, 'n_tokens_neox': 2415, 'n_words': 764, 'path': 'K-Means.ipynb', 'repo_name': 'sescapa/EN-Machine-Learning', 'size': '6758'}