<jupyter_start><jupyter_text>AbstractThis notebook compares the candidate mechanism perturbation amplitude at multiple disease stages in Alzheimer's Disease experiments, using the PyBEL web service<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import os
import seaborn as sns
from pandas.plotting import andrews_curves, parallel_coordinates
from sklearn.cluster import KMeans
from sklearn import preprocessing
import time
#%config InlineBackend.figure_format = 'svg'
%matplotlib inline
np.random.seed(5)
time.asctime()
downloads = os.path.join(os.path.expanduser('~'), 'Downloads')

early = os.path.join(downloads, '1.csv')
moderate = os.path.join(downloads, '2.csv')
severe = os.path.join(downloads, '3.csv')
scaler = preprocessing.StandardScaler()
early_df = pd.read_csv(early).rename(index=str, columns={"avg": "EarlyAvg"})
moderate_df = pd.read_csv(moderate).rename(index=str, columns={"avg": "ModerateAvg"})
severe_df = pd.read_csv(severe).rename(index=str, columns={"avg": "SevereAvg"})

df = pd.concat(
    [
        early_df[['Namespace', 'Name', 'EarlyAvg']], 
        moderate_df['ModerateAvg'], 
        severe_df['SevereAvg']
    ], 
    axis=1
)

df = df[df['EarlyAvg'].notnull()]
df = df[df['EarlyAvg'] != 0]

cols = ['EarlyAvg', 'ModerateAvg', 'SevereAvg']

df.to_csv(os.path.join(os.path.expanduser('~'), 'Desktop', 'time_series_cmpa.csv'))
df.head()
sns.pairplot(df[['EarlyAvg', 'ModerateAvg', 'SevereAvg']])
plt.show()<jupyter_output><empty_output><jupyter_text>Use the pearson correlation over the time series<jupyter_code>corr_df = df[cols].T.corr()<jupyter_output><empty_output><jupyter_text>Clustering reveals 3 general patterns of biological processes throughout the progressions.<jupyter_code>cg = sns.clustermap(corr_df)
plt.setp(cg.ax_heatmap.yaxis.get_majorticklabels(), rotation=0)
plt.savefig(os.path.join(os.path.expanduser('~'), 'Desktop', 'time_series_clustering_ad.pdf'))
plt.show()<jupyter_output><empty_output><jupyter_text>Assign classes based on a simple k-means clustering.<jupyter_code>km = KMeans(n_clusters=5)
km.fit(df[cols])

df['label'] = km.labels_<jupyter_output><empty_output><jupyter_text>Parallel coordinates immediately reveal the groups of patterns in relation of each mechanism to the disease progressions. Andrews curves use fourier analysis to reveal further patterns in the frequency domain.<jupyter_code>parallel_coordinates(df[['EarlyAvg', 'ModerateAvg', 'SevereAvg', 'label']], 'label')
plt.savefig(os.path.join(os.path.expanduser('~'), 'Desktop', 'time_series_pc.pdf'))<jupyter_output><empty_output><jupyter_text>Retry the whole analysis but min/max normalize each column first<jupyter_code>df_norm = df[cols].apply(lambda x: (x - np.min(x)) / (np.max(x) - np.min(x)))

norm_corr_df = df_norm[cols].T.corr()
sns.pairplot(df_norm[['EarlyAvg', 'ModerateAvg', 'SevereAvg']])
plt.show()
#cg = sns.clustermap(norm_corr_df)
#plt.setp(cg.ax_heatmap.yaxis.get_majorticklabels(), rotation=0)
#plt.show()
km_norm = KMeans(n_clusters=6)
km_norm.fit(df[cols])

df_norm['label'] = km_norm.labels_
plt.title('Parallel Coordinates on Normalized Data')
parallel_coordinates(df_norm, 'label')
plt.show()<jupyter_output><empty_output><jupyter_text>Class `0` contains candidate mechanisms whose CMPA scores are neither significant nor do they change much over time. The other class members are enumerated below.<jupyter_code>df[df['label'] == 0]
df[df['label'] == 1]
df[df['label'] == 2]
df[df['label'] == 3]
df[df['label'] == 4]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14814814814814814, 'fraction_numerical': 0.004806333050607859, 'mean_word_length': 7.324705882352941, 'pattern_counts': {'":': 3, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '255399', 'license': 'Apache-2.0', 'n_tokens_mistral': 1332, 'n_tokens_neox': 1258, 'n_words': 255, 'path': 'algorithms/Time Series CMPA.ipynb', 'repo_name': 'djinnome/pybel-notebooks', 'size': '3537'}