<jupyter_start><jupyter_text>01장. 클러스터링 3. 계층적 클러스터링<jupyter_code>from scipy.cluster.hierarchy import linkage, dendrogram
import numpy as np
import matplotlib.pylab as plt
import matplotlib as mpl
np.set_printoptions(precision=5, suppress=True) 

np.random.seed(4711)  # for repeatability of this tutorial
a = np.random.multivariate_normal([10, 0], [[3, 1], [1, 4]], size=[100,])
b = np.random.multivariate_normal([0, 20], [[3, 1], [1, 4]], size=[50,])
X = np.concatenate((a, b),)
print(X.shape)

plt.scatter(X[:,0], X[:,1], s=50)
plt.show()
X<jupyter_output><empty_output><jupyter_text>linkage    - linkage(데이터, '데이터 거리 측정 방식')    - 계층적 클러스터링 해준다.<jupyter_code>Z = linkage(X, 'ward')<jupyter_output><empty_output><jupyter_text>- 클러스터링 이후 4개의 특성을 output으로 반환하는데    - 클러스터 넘버 2개, 두개의 클러스터 간 거리, 클러스터의 cardinality 원소개수<jupyter_code>Z[:20]
idxs = [33, 68, 62, 82, 63, 98]
plt.figure(figsize=(10, 10))
plt.scatter(X[:,0], X[:,1], s=50)
plt.scatter(X[idxs,0], X[idxs,1], c='r', s=100) 
plt.show()<jupyter_output><empty_output><jupyter_text>아래의 데이터만 빨갛게 표시<jupyter_code>X[idxs,0]
X[idxs,1]<jupyter_output><empty_output><jupyter_text>dendrogram    - 계층적 클러스터링 시각화 해준다.<jupyter_code>mpl.rcParams["font.family"] = 'DejaVu Sans'
plt.figure(figsize=(10,30))
ax = plt.subplot(111)
dendrogram(Z, leaf_font_size=10, orientation='right')
plt.title('Hierarchical Clustering Dendrogram')
plt.xlabel('sample index')
plt.ylabel('distance')
ax.set_xlim(xmin=0.03)
ax.set_xscale('log')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17674418604651163, 'fraction_numerical': 0.04916943521594684, 'mean_word_length': 6.968253968253968, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '284837', 'license': 'MIT', 'n_tokens_mistral': 805, 'n_tokens_neox': 847, 'n_words': 122, 'path': '10_clustering/03_hierarchial_clustering.ipynb', 'repo_name': 'hyelansgithub/fastcampus-math', 'size': '1505'}