<jupyter_start><jupyter_text>Example: Fuzzy c-means clustering and silhouette coefficients<jupyter_code>import numpy as np
import matplotlib.pyplot as plt

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.metrics import accuracy_score
from sklearn.metrics import silhouette_samples

from dskit.clustering import FuzzyCMeans
from dskit.graphics import silhouette_plot

%matplotlib inline
RANDOM_STATE = 0<jupyter_output><empty_output><jupyter_text>Generating dataGenerate sample data<jupyter_code>X, y = make_blobs(
    n_samples=150,
    n_features=2,
    centers=3,
    cluster_std=0.5,
    shuffle=True,
    random_state=RANDOM_STATE
)<jupyter_output><empty_output><jupyter_text>Visual inspection of sample data<jupyter_code>plt.figure(figsize=(10, 6))
plt.scatter(X[:, 0], X[:, 1], 
            c='white', marker='o', 
            edgecolor='black', s=50)
plt.grid()
plt.tight_layout()<jupyter_output><empty_output><jupyter_text>Silhouette analysisApply the k-means and fuzzy-c-means algorithm to the sample data. Intentionally specify one cluster too many compared to visual inspection of sample data<jupyter_code># Setup
n_clusters = 4
max_iter = 200
tol = 1e-04
random_state = 0
kmeans = KMeans(n_clusters=n_clusters,
                max_iter=max_iter,
                tol=tol,
                random_state=RANDOM_STATE)

kmeans_pred = kmeans.fit_predict(X)
fuzzy = FuzzyCMeans(n_clusters=n_clusters,
                    max_iter=max_iter,
                    tol=tol,
                    random_state=RANDOM_STATE)

fuzzy_pred = fuzzy.fit_predict(X)<jupyter_output><empty_output><jupyter_text>Generate silhouette plots for each cluster<jupyter_code>plt.figure(figsize=(10, 12))

plt.subplot(2, 1, 1)
silhouette_plot(X, kmeans_pred)
plt.title('K-means clustering')

plt.subplot(2, 1, 2)
silhouette_plot(X, fuzzy_pred)
plt.title('Fuzzy c-means clustering')

plt.tight_layout()<jupyter_output><empty_output><jupyter_text>Model comparisonComparing the cluster labels assigned by each algorithm to the input data points with the generated sample data labels.<jupyter_code>kmeans_score = accuracy_score(y, kmeans_pred)
fuzzy_score = accuracy_score(y, fuzzy_pred)

print('K-means score: {}\nFuzzy c-means score: {:.2f}'.format(kmeans_score, 
                                                              fuzzy_score))<jupyter_output>K-means score: 0.14
Fuzzy c-means score: 0.63
{'fraction_non_alphanumeric': 0.09267687215556475, 'fraction_numerical': 0.016549441456350848, 'mean_word_length': 4.325991189427313, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '324507', 'license': 'MIT', 'n_tokens_mistral': 902, 'n_tokens_neox': 810, 'n_words': 174, 'path': 'examples/fuzzy_c_means.ipynb', 'repo_name': 'GSEL9/data-science-kit', 'size': '2417'}