<jupyter_start><jupyter_text>Load datasets used in the manuscript "A Swiss-Army Knife for Hierarchical Modeling of Biological Systems" (Yu et al.)<jupyter_code>import ddot
from ddot import Ontology<jupyter_output><empty_output><jupyter_text>Gene-disease associations from Monarch Initiative<jupyter_code># Retrieve a table of gene-disease associations from the Monarch Initiative (reformatted and stored on NDEx)
monarch, _ = ddot.ndex_to_sim_matrix(
    ndex_url=ddot.MONARCH_DISEASE_GENE_SLIM_URL,
    input_fmt='cx',
    output_fmt='sparse')
monarch.head()
# Example: get the known genes for "Caffey Disease"
seed = monarch.loc[monarch['disease']=='caffey_disease', 'gene'].tolist()
print('Seed:', seed)<jupyter_output>Seed: ['COL1A1', 'A4GALT']<jupyter_text>Human gene-gene similarity network<jupyter_code># Install the simplejson package (it is recommend you run this in a separate bash terminal, not in this Jupyter notebook. If you want to use a conda virtual environment, then you first need to activate the environment)
! pip install simplejson
## Download human gene-gene similarity network from NDEx
## -- WARNING: This network is very large (19,009-by-19,009 matrix).
## -- *** Requires ~6 GB of RAM to download and process ***
## -- *** ~4 GB of data is downloaded from NDEx, which takes ~10 min for a fast internet connection ***
## -- This requires the simplejson package (see previous cell)
sim, sim_names = ddot.ndex_to_sim_matrix(
    ndex_url=ddot.HUMAN_GENE_SIMILARITIES_URL,
    input_fmt='cx_matrix',
    output_fmt='matrix')

import pandas as pd
sim = pd.DataFrame(sim, columns=sim_names, index=sim_names)

sim.head()<jupyter_output>NDEx download and CX parse time (sec): 356.6890811920166
Dim: (19009, 19009)
Bytes: 2890736648
Iterate through CX and construct array time (sec): 14.676801204681396<jupyter_text>The Gene Ontology<jupyter_code># Read Gene Ontology from NDEx. 
# -- This version has been pre-processed to contain a non-redundant set of GO terms and connections that are relevant to human genes (see Process_the_Gene_Ontology.ipynb) 
go_human = Ontology.from_ndex(ddot.GO_HUMAN_URL)
print(go_human)<jupyter_output>19015 genes, 19343 terms, 215488 gene-term relations, 36362 term-term relations
node_attributes: ['Branch', 'Vis:Shape', 'name', 'Vis:Fill Color', 'Vis:Border Paint', 'Term_Description']
edge_attributes: ['Vis:Visible']<jupyter_text>Fanconi Anemia gene ontology (FanGO)<jupyter_code>fango = Ontology.from_ndex(ddot.FANGO_URL)
print(fango)<jupyter_output>349 genes, 110 terms, 349 gene-term relations, 109 term-term relations
node_attributes: ['Hidden', 'Similarity_to_seed', 'Seed', 'is_collect_node', 'Display:Aligned_Term', 'Display:Aligned_FDR', 'Display:Aligned_Term_Description', 'name', 'Vis:Border Paint', 'Display:Parent weight', 'Display:Aligned_Similarity', 'Parent weight', 'Original_Name', 'ndex:internalLink', 'Aligned_Term', 'Aligned_Term_Description', 'Aligned_FDR', 'Vis:Shape', 'Vis:Fill Color', 'Aligned_Similarity']
edge_attributes: ['Vis:EDGE_TARGET_ARROW_SHAPE', 'Vis:EDGE_SOURCE_ARROW_SHAPE', 'CLIXO_score', 'Vis:Visible']<jupyter_text>Other disease gene ontologies (based on gene-disease associations in Monarch Initiative)<jupyter_code>import pandas as pd
df = pd.read_table('disease_gene_ontologies.txt', header=0, index_col=False)
df = df.set_index('Disease')
df.head()
# Example: get the URL to view the disease "hydronephrosis" on HiView
print(df.loc['hydronephrosis', 'HiView_URL'])<jupyter_output>http://hiview.ucsd.edu/1b21bc44-f775-11e8-aaa6-0ac135e8bacf?type=public&server=http://public.ndexbio.org
{'fraction_non_alphanumeric': 0.1133761519128735, 'fraction_numerical': 0.033789444289304665, 'mean_word_length': 7.159453302961276, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '279034', 'license': 'MIT', 'n_tokens_mistral': 1335, 'n_tokens_neox': 1201, 'n_words': 322, 'path': 'examples/Load_example_datasets.ipynb', 'repo_name': 'michaelkyu/ontology_cx', 'size': '3581'}