<jupyter_start><jupyter_text>Proteins example recreated in python from https://rstudio-pubs-static.s3.amazonaws.com/33876_1d7794d9a86647ca90c4f182df93f0e8.html and http://nbviewer.jupyter.org/github/OxanaSachenkova/hclust-python/blob/master/hclust.ipynb<jupyter_code>import numpy as np
from numpy import genfromtxt
data = genfromtxt('http://www.biz.uiowa.edu/faculty/jledolter/DataMining/protein.csv',delimiter=',',names=True,dtype=float)<jupyter_output><empty_output><jupyter_text>note numpy also has recfromcsv() and pandas can read_csv, with pandas DF.values giving a numpy array<jupyter_code>len(data)
len(data.dtype.names)
data.dtype.names
type(data)
data
data_array = data.view((np.float, len(data.dtype.names)))
data_array
data_array = data_array.transpose()
print(data_array)

data_array[1:10]<jupyter_output><empty_output><jupyter_text>Samples clustering using scipyFirst, we'll implement the clustering using scipy modules<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from scipy.spatial.distance import pdist, squareform
from scipy.cluster.hierarchy import linkage, dendrogram
data_dist = pdist(data_array[1:10]) # computing the distance
data_link = linkage(data_dist) # computing the linkage
dendrogram(data_link,labels=data.dtype.names)
plt.xlabel('Samples')
plt.ylabel('Distance')
plt.suptitle('Samples clustering', fontweight='bold', fontsize=14);
plt.show()
# Compute and plot first dendrogram.
fig = plt.figure(figsize=(8,8))
# x ywidth height
ax1 = fig.add_axes([0.05,0.1,0.2,0.6])
Y = linkage(data_dist, method='single')
Z1 = dendrogram(Y, orientation='right',labels=data.dtype.names) # adding/removing the axes
ax1.set_xticks([])

# Compute and plot second dendrogram.
ax2 = fig.add_axes([0.3,0.71,0.6,0.2])
Z2 = dendrogram(Y)
ax2.set_xticks([])
ax2.set_yticks([])

#Compute and plot the heatmap
axmatrix = fig.add_axes([0.3,0.1,0.6,0.6])
idx1 = Z1['leaves']
idx2 = Z2['leaves']
D = squareform(data_dist)
D = D[idx1,:]
D = D[:,idx2]
im = axmatrix.matshow(D, aspect='auto', origin='lower', cmap=plt.cm.YlGnBu)
axmatrix.set_xticks([])
axmatrix.set_yticks([])

# Plot colorbar.
axcolor = fig.add_axes([0.91,0.1,0.02,0.6])
plt.colorbar(im, cax=axcolor)
plt.show()<jupyter_output><empty_output><jupyter_text>the fastcluster module http://math.stanford.edu/~muellner/fastcluster.html?section=0<jupyter_code>! pip install fastcluster
from fastcluster import *
%timeit data_link = linkage(data_array[1:10], method='single', metric='euclidean', preserve_input=True)
dendrogram(data_link,labels=data.dtype.names)
plt.xlabel('Samples')
plt.ylabel('Distance')
plt.suptitle('Samples clustering', fontweight='bold', fontsize=14);
plt.show()<jupyter_output>The slowest run took 5.77 times longer than the fastest. This could mean that an intermediate result is being cached.
10000 loops, best of 3: 45.1 µs per loop
{'fraction_non_alphanumeric': 0.13953488372093023, 'fraction_numerical': 0.036645525017618044, 'mean_word_length': 7.276967930029154, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '434299', 'license': 'Apache-2.0', 'n_tokens_mistral': 1138, 'n_tokens_neox': 1065, 'n_words': 205, 'path': 'Hierarchical+Clustering (1).ipynb', 'repo_name': 'sayanb3/python_for_datascience', 'size': '2838'}