<jupyter_start><jupyter_text>Loading useful librariesMore details about them can be found in `readme.md`<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.datasets import load_iris<jupyter_output><empty_output><jupyter_text>Loading dataset and converting it to dataframe using Pandas<jupyter_code>data = load_iris()
df = pd.DataFrame(data.data, columns=data.feature_names)
df.head()<jupyter_output><empty_output><jupyter_text>Simple EDA<jupyter_code>cols_to_delete = df.columns[df.isna().sum()/len(df) >= .999]
cols_to_delete
# example of how to count unique values in a column
print(pd.crosstab(index=df["sepal length (cm)"], columns="count") )
count_sepal_length = (df['sepal length (cm)'] == 5.0).sum()
print(count_sepal_length)
# plotting works a lot better with np arrays, so we convert dimensions to nparrays to plot
x1 = np.array(df['sepal length (cm)'])
x2 = np.array(df['sepal width (cm)'])
plt.plot()
plt.title('Sepal Length vs. Sepal Width Scatter Plot')
plt.xlabel('Sepal length')
plt.ylabel('Sepal width')
plt.scatter(x1, x2)
plt.show()<jupyter_output><empty_output><jupyter_text>Implementing K-Means Elbow method to determine optimal K value<jupyter_code>x = df.iloc[:, [0,1,2,3]].values
Error =[]
for i in range(1, 11):
    kmeans = KMeans(n_clusters = i).fit(x)
    kmeans.fit(x)
    Error.append(kmeans.inertia_)
import matplotlib.pyplot as plt
plt.plot(range(1, 11), Error, marker = 'o')
plt.title('Elbow method')
plt.xlabel('No of clusters')
plt.ylabel('Error')
plt.show()<jupyter_output><empty_output><jupyter_text>The plot above shows us that the optimal number of clusters is 3. We will use that to train our model<jupyter_code>kmeans = KMeans(n_clusters=3).fit(x)
y_kmeans = kmeans.fit_predict(x)
y_kmeans
kmeans.cluster_centers_
x = np.array(list(zip(x1, x2)))
colors = ['b', 'g', 'r']
markers = ['x', 'o', 'v']
plt.ylabel('Length')


plt.scatter(kmeans.cluster_centers_[:,0], kmeans.cluster_centers_[:, 1], s = 200, c= 'yellow', label = 'Centroids')

for i,j in enumerate(kmeans.labels_):
    plt.plot(x1[i], x2[i], color=colors[j], marker=markers[j])
plt.xlabel('Width')
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Other plot options<jupyter_code>plt.scatter(x[:, 0], x[:, 1], c=y_kmeans, cmap='rainbow')<jupyter_output><empty_output><jupyter_text>Example Cluster Prediction<jupyter_code>kmeans.predict(np.array([5.3, 2.9, 1.5, 2.3]).reshape(1,-1) )[0]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1532033426183844, 'fraction_numerical': 0.017111022682053324, 'mean_word_length': 6.955696202531645, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 30, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '207886', 'license': 'Apache-2.0', 'n_tokens_mistral': 1021, 'n_tokens_neox': 969, 'n_words': 213, 'path': 'clustering/KMeans.ipynb', 'repo_name': 'Headstorm/data-science-toolbox', 'size': '2513'}