<jupyter_start><jupyter_text>Hierarchical ClusteringIn this notebook we will give a basic example of how agglomerative hierarchical cluster works. We use scipy and sklearn libraries.<jupyter_code>from sklearn.metrics import normalized_mutual_info_score
import matplotlib.pyplot as plt
from scipy.cluster.hierarchy import dendrogram, linkage, fcluster
from sklearn.datasets.samples_generator import make_blobs
import numpy as np<jupyter_output><empty_output><jupyter_text>Generating Sample data `make_blobs` is used to generate sample data where: `n_samples`        :  the total number of points equally divided among clusters.`centers`           :  the number of centers to generate, or the fixed center locations.`n_features`     : the number of features for each sample. `random_state`: determines random number generation for dataset creation.This function returns two outputs: `X`:   the generated samples. `y`:   The integer labels for cluster membership of each sample. Then we use `plt.scatter` to plot the data points in the figure below.<jupyter_code>X, y = make_blobs(n_samples=90, centers=4, n_features=3, random_state=4)
plt.scatter(X[:, 0], X[:, 1])
plt.show()<jupyter_output><empty_output><jupyter_text>Performing Hierarchical clustering: In this part, we are performing agglomerative hierarchical clustering using linkage function from scipy library:: `method`: is the linkage method, 'single' means the linkage method will be single linkage method. `metric`: is our similarity metric, 'euclidean' means the metric will be euclidean distance. "A  `(n-1)` by 4 matrix `Z` is returned. At the -th iteration, clusters with indices `Z[i, 0]` and `Z[i, 1]` are combined to form cluster with index `(n+i)` . A cluster with an index less than `n` corresponds to one of the `n` original observations. The distance between clusters `Z[i, 0]` and `Z[i, 1]` is given by `Z[i, 2]`. The fourth value `Z[i, 3]` represents the number of original observations in the newly formed cluster.The following linkage methods are used to compute the distance `d(s,t)`between two clusters `s`and `t`. The algorithm begins with a forest of clusters that have yet to be used in the hierarchy being formed. When two clusters `s` and `t`from this forest are combined into a single cluster `u`, `s`and `t` are removed from the forest, and `u` is added to the forest. When only one cluster remains in the forest, the algorithm stops, and this cluster becomes the root.A distance matrix is maintained at each iteration. The `d[i,j]`` entry corresponds to the distance between cluster `ii` and `j` in the original forest.At each iteration, the algorithm must update the distance matrix to reflect the distance of the newly formed cluster u with the remaining clusters in the forest."For more details check the docmentation of linkage: https://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.hierarchy.linkage.html<jupyter_code>Z = linkage(X, method="single", metric="euclidean")
print(Z.shape)
Z<jupyter_output>(89, 4)<jupyter_text>Plotting dendrogram The dedrogram function from scipy is used to plot dendrogram: *   On the `x` axis we see the indexes of our samples. *   On the `y` axis we see the distances of our metric ('Euclidean').<jupyter_code>plt.figure(figsize=(25, 10))
plt.title("Hierarchical Clustering Dendrogram")
plt.xlabel("Samples indexes")
plt.ylabel("distance")
dendrogram(Z, leaf_rotation=90., leaf_font_size=8. ) 
plt.show()<jupyter_output><empty_output><jupyter_text>Retrive the clusters`fcluster` is used to retrive clusters with some level of distance. The number two determines the distance in which we want to cut the dendrogram. The number of crossed line is equal to number of clusters.<jupyter_code>cluster = fcluster(Z, 2, criterion="distance")
cluster<jupyter_output><empty_output><jupyter_text>Plotting ClustersPlotting the final result. Each color represents a different cluster (four clusters in total).<jupyter_code>plt.figure(figsize=(10, 8))
plt.scatter(X[:, 0], X[:, 1], c=cluster, cmap="Accent")
plt.savefig("clusters.png")
plt.show()<jupyter_output><empty_output><jupyter_text>Evaluting clusters: Finally we will use Normalized Mutual Information (NMI) score to evaluate our clusters. Mutual information is a symmetric measure for the degree of dependency between the clustering and the manual classification. When NMI value is close to one, it indicates high similarity between clusters and actual labels. But if it was close to zero, it indicates high dissimilarity between them.<jupyter_code>normalized_mutual_info_score(y, cluster)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08306153511632963, 'fraction_numerical': 0.006740595781691672, 'mean_word_length': 5.804733727810651, 'pattern_counts': {'":': 0, '<': 28, '<?xml version=': 0, '>': 28, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '507297', 'license': 'MIT', 'n_tokens_mistral': 1374, 'n_tokens_neox': 1289, 'n_words': 562, 'path': '4-assets/BOOKS/Jupyter-Notebooks/Overflow/hierarchical_clustering.ipynb', 'repo_name': 'impastasyndrome/Lambda-Resource-Static-Assets', 'size': '4599'}