<jupyter_start><jupyter_text>k means clustering is an unsupervised machine learning algorithmThree key features of k-means that make it efficient are often regarded as its biggest drawbacks:Euclidean distance is used as a metric and variance is used as a measure of cluster scatter.The number of clusters k is an input parameter: an inappropriate choice of k may yield poor results. That is why, when performing k-means, it is important to run diagnostic checks for determining the number of clusters in the data set.Convergence to a local minimum may produce counterintuitive ("wrong") results Source: https://en.wikipedia.org/wiki/K-means_clustering  find the number of clusters using the elbow method<jupyter_code>## scale the data

mms = MinMaxScaler()
data.loc[:, data.columns != 'class'] = mms.fit_transform(data.loc[:, data.columns != 'class'])

data_elb = data.loc[:, data.columns != 'class']

sse = {}
for k in range(1, 20):
    kmeans = KMeans(n_clusters=k, max_iter=1000).fit(data_elb)
    data_elb["clusters"] = kmeans.labels_
    #print(data["clusters"])
    sse[k] = kmeans.inertia_ # Inertia: Sum of distances of samples to their closest cluster center
plt.figure()
plt.plot(list(sse.keys()), list(sse.values()))
plt.xlabel("Number of cluster")
plt.ylabel("SSE")
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1039453717754173, 'fraction_numerical': 0.005311077389984826, 'mean_word_length': 5.311004784688995, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '550803', 'license': 'MIT', 'n_tokens_mistral': 416, 'n_tokens_neox': 389, 'n_words': 148, 'path': 'Section 6/6.1.ipynb', 'repo_name': 'PacktPublishing/Scikit-learn-Recipes', 'size': '1318'}