<jupyter_start><jupyter_text>Dataset labelled by experts to be used as test data<jupyter_code>experts = pd.read_csv('../data/input/expert_resolved_all.csv')
experts.sample()
experts = experts.rename(columns={'drug_name':'Drug_name',
                        'disease_name':'disease', 'context':'Context',
                        'do_id':'DO_ID','drug_id':'DB_ID',
                        'label_id':'Label_ID', 'expert_consensus':'relation'})
#experts[['DB_ID', 'DO_ID','disease', 'Label_ID', 'Set_ID','relation', 'Drug_name', 'Context','Section']]

experts.DB_ID='Compound::'+experts.DB_ID
experts.DO_ID= 'Disease::DOID:'+experts.DO_ID.str[5:]
experts.head()
experts.relation = experts.relation =='Indication: Treatment'
experts.relation.sum()
df_het = pd.read_csv('../data/input/hetionet_ground_truth.csv')
#df_het= df_het.append(experts[['DB_ID','DO_ID']])
df_het.DB_ID='Compound::'+df_het.DB_ID
df_het.DO_ID= 'Disease::'+df_het.DO_ID
df_het.head()
from sklearn.model_selection import train_test_split
df_het_train, df_het_test =train_test_split(df_het, shuffle=False, test_size=0.2)
df_het_train.shape
indications_het =set()
experts_dict = {}
for i,row in df_het_train.iterrows():
    #print (row['a'], row['b'])
    indications_het.add((str(row['DB_ID']),str(row['DO_ID'])))
    #break
len(indications_het)
('Compound::DB08906', 'Disease::DOID:2841') in indications_het
whole_dr_di = []
for dr in df_het.DB_ID.unique():
    for di in df_het.DO_ID.unique():
        if (dr, di) in indications_het:
            #print ((dr, di))
            whole_dr_di.append([dr, di, 1])
        else:
            whole_dr_di.append([dr, di, 0])
df_train = pd.DataFrame(whole_dr_di, columns=['DB_ID', 'DO_ID','label'])<jupyter_output><empty_output><jupyter_text>Define KG Rules as Labeling Functions in SnorkelSnorkel is data labeling framework where multiple heuristic and programatic rules can be combined to assign a label to training data<jupyter_code># Labels to be assigned 
POSITIVE = 1 # positive that the drug indication is recommendable
NEGATIVE = 0 # negative that the drug indication is recommendable
ABSTAIN = -1 # not sure whether the drug indication is recommendable or not<jupyter_output><empty_output><jupyter_text>Embedding  TransE Embeddings (Pre-Trained)<jupyter_code>import numpy as np
import csv
from sklearn.metrics.pairwise import cosine_similarity

import torch.nn.functional as fn
import torch as th

node_emb = np.load('../embed/DRKG_TransE_l2_entity.npy')
relation_emb = np.load('../embed/DRKG_TransE_l2_relation.npy')

print(node_emb.shape)
print(relation_emb.shape)
entity2id = {}
id2entity = {}
with open("../embed/entities.tsv", newline='', encoding='utf-8') as csvfile:
    reader = csv.DictReader(csvfile, delimiter='\t', fieldnames=['entity','id'])
    for row_val in reader:
        id = row_val['id']
        entity = row_val['entity']

        entity2id[entity] = int(id)
        id2entity[int(id)] = entity

print("Number of entities: {}".format(len(entity2id)))
relation2id = {}
id2relation = {}
with open("../embed/relations.tsv", newline='', encoding='utf-8') as csvfile:
    reader = csv.DictReader(csvfile, delimiter='\t', fieldnames=['relation','id'])
    for row_val in reader:
        id = row_val['id']
        relation = row_val['relation']
        #print (relation, id)
        relation2id[relation] = int(id)
        id2relation[int(id)] = relation

print("Number of relations: {}".format(len(relation2id)))
node_emb[entity2id['Compound::DB00970']][:10]
i =0
for e in  entity2id:
    if 'Compound::DB' in e or 'Disease::DOID:' in e:
        i+=1
print (i)

with open('../embed/DRGK_entity_embedding.txt','w') as fw:
    fw.write(str(i)+' '+str(node_emb.shape[1])+'\n')
    for e in  entity2id:
        if 'Compound::DB' in e or 'Disease::DOID:' in e:
            fw.write(e+' '+" ".join( [str(v) for v in  node_emb[entity2id[e]]] )+'\n')<jupyter_output>10678<jupyter_text>RDF2VEC Emebdding<jupyter_code>from gensim.models import KeyedVectors
word_vectors = KeyedVectors.load_word2vec_format('../embed/DRGK_entity_embedding.txt', binary=False)
word_vectors.most_similar('Disease::DOID:2841')
word_vectors.similar_by_vector(word_vectors['Compound::DB00970'])
# Use Embedding model in the labeling function<jupyter_output><empty_output><jupyter_text>A probablistic rule is a rule that contains uncertainty about reasoningBased on the guilt-by-association principle (e.g. Barabasi et al., 2011;Chiang and Butte, 2009), new disease–drug relationships can be inferredthrough existing relationships between similar diseases and similar drugs. Example: If drug ${dr}$ treats disease ${ds}$ and disease ${ds}$ is similar to disease ${ds'}$ then drug ${dr}$ treats disease ${ds'}$*  ${?dr}$ CtD ${?ds}$   ^  ${?ds}$  DrD ${?ds'}$  => ${?dr}$ CtD ${?ds'}$<jupyter_code>from snorkel.labeling import labeling_function
def rule1_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    #print (drug, disease)
    # search all indications treating the same disease
    # if the drug that is similar to the one in the known indication,
    # return 'INDICATION' 
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        r2 = relation_emb[relation2id['Hetionet::DrD::Disease:Disease']]
        a = node_emb[dr_idx]
        b = node_emb[ds_idx]
        similarEntity = word_vectors.similar_by_vector(a + r1 + r2)
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Translation based probablistic rule as labeling function$if (h, r, t) => h + r = t  $, vector norm of difference between head entity plus relation and tail entity in embedding space should be zero* we can simply assess the correctness of a triple (h, r, t) bychecking the plausibility score of the embedding vectors of h, r and t.<jupyter_code>@labeling_function()
def transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    #print (drug, disease)
    # search all indications treating the same disease
    # if the drug that is similar to the one in the known indication,
    # return 'INDICATION' 
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        similarEntity = word_vectors.similar_by_vector((node_emb[dr_idx] + r), topn=10)
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN 
relation2id<jupyter_output><empty_output><jupyter_text>Probablistic Rule using TransE Embedding: Rule1: If drug $?dr$ treats disease $?f$ and disease $?f$ resembles disease $?ds$ then drug $?dr$  treats  $?ds$  $$dr + r1 = f,   f + r2 = ds  => dr + r1 + r2 = ds, $$$$r1= treats , r2= resembles$$$$ sim = cosine (dr, ds) $$<jupyter_code>@labeling_function()
def rule1_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    #print (drug, disease)
    # search all indications treating the same disease
    # if the drug that is similar to the one in the known indication,
    # return 'INDICATION' 
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        r2 = relation_emb[relation2id['Hetionet::DrD::Disease:Disease']]
        a = node_emb[dr_idx]
        b = node_emb[ds_idx]
        similarEntity = word_vectors.similar_by_vector(a + r1 + r2)
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Rule 2:If drug $?dr$ binds gene $?g$ and disease $?ds$ associates gene $?g$ then drug $?dr$  treats  $?ds$   Compound–binds–Gene–associates–Disease => Compound–treats–Disease<jupyter_code>@labeling_function()
def rule2_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    #print (drug, disease)
    # search all indications treating the same disease
    # if the drug that is similar to the one in the known indication,
    # return 'INDICATION' 
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CbG::Compound:Gene']]
        r2 = relation_emb[relation2id['Hetionet::DaG::Disease:Gene']]
        a = node_emb[dr_idx]
        similarEntity = word_vectors.similar_by_vector(a + r1 -r2)
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Rule 3:  Compound–resemble–Compound–Treat–Disease => Compound–treats–Disease<jupyter_code>@labeling_function()
def rule3_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    #print (drug, disease)
    # search all indications treating the same disease
    # if the drug that is similar to the one in the known indication,
    # return 'INDICATION' 
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        r2 = relation_emb[relation2id['Hetionet::CrC::Compound:Compound']]
        a = node_emb[dr_idx]
        b = node_emb[ds_idx]
        similarEntity = word_vectors.similar_by_vector(a + r2 + r1)
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Rule 4Compound-include(-1)-Pharmacologic Class-include-Compound--treat-disease<jupyter_code>@labeling_function()
def rule4_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::PCiC::Pharmacologic Class:Compound']]
        r2 = relation_emb[relation2id['Hetionet::PCiC::Pharmacologic Class:Compound']]
        r3 = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        similarEntity = word_vectors.similar_by_vector((node_emb[dr_idx] -r1 + r2 +r3))
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Rule 5Compound-resembles-Compound-resembles-Compound-treats-Disease<jupyter_code>@labeling_function()
def rule5_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CrC::Compound:Compound']]
        r2 = relation_emb[relation2id['Hetionet::CrC::Compound:Compound']]
        r3 = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        similarEntity = word_vectors.similar_by_vector((node_emb[dr_idx] + r1 + r2 +r3))
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Rule 6Compound-palliates-Disease-palliates(-1)-Compound-treats-Disease<jupyter_code>@labeling_function()
def rule6_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CpD::Compound:Disease']]
        r2 = relation_emb[relation2id['Hetionet::CpD::Compound:Disease']]
        r3 = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        similarEntity = word_vectors.similar_by_vector((node_emb[dr_idx] + r1 - r2 +r3))
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Rule 7Compound-binds-Gene-binds(-1)-Compound-treats-Disease<jupyter_code>@labeling_function()
def rule7_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CbG::Compound:Gene']]
        r2 = relation_emb[relation2id['Hetionet::CbG::Compound:Gene']]
        r3 = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        similarEntity = word_vectors.similar_by_vector((node_emb[dr_idx] + r1 - r2 +r3))
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Rule 8Compound-causes-Side Effect-causes(1)-Compound-treats-Disease<jupyter_code>@labeling_function()
def rule8_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CcSE::Compound:Side Effect']] 
        r2 = relation_emb[relation2id['Hetionet::CcSE::Compound:Side Effect']] 
        r3 = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        similarEntity = word_vectors.similar_by_vector((node_emb[dr_idx] + r1 - r2 +r3))
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Rule 9 Compound-resembles-Compound-binds-Gene-associates(-1)-Disease<jupyter_code>@labeling_function()
def rule9_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CrC::Compound:Compound']] 
        r2 = relation_emb[relation2id['Hetionet::CbG::Compound:Gene']] 
        r3 = relation_emb[relation2id['Hetionet::DaG::Disease:Gene']]
        similarEntity = word_vectors.similar_by_vector((node_emb[dr_idx] + r1 + r2 - r3))
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN<jupyter_output><empty_output><jupyter_text>Rule 10Compound-binds-Gene-expresses(1)-Anatomy-localizes(1)-Diseasecosine_similarity(dr+r1-r2-r3, ds)<jupyter_code>@labeling_function()
def rule10_transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r1 = relation_emb[relation2id['Hetionet::CbG::Compound:Gene']] 
        r2 = relation_emb[relation2id['Hetionet::AeG::Anatomy:Gene']] 
        r3 = relation_emb[relation2id['Hetionet::DlA::Disease:Anatomy']]
        similarEntity = word_vectors.similar_by_vector((node_emb[dr_idx] + r1 - r2 - r3))
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN 
@labeling_function()
def transe_sim(x):
    drug= x.DB_ID
    disease = x.DO_ID
    #print (drug, disease)
    # search all indications treating the same disease
    # if the drug that is similar to the one in the known indication,
    # return 'INDICATION' 
    score = 0.0
    if drug in entity2id and disease in entity2id:
        dr_idx = entity2id[drug]
        ds_idx = entity2id[disease]
        r = relation_emb[relation2id['Hetionet::CtD::Compound:Disease']]
        similarEntity = word_vectors.similar_by_vector((node_emb[dr_idx] + r), topn=10)
        for en,sim in similarEntity:
            if en == disease:
                return POSITIVE
    return ABSTAIN 
from snorkel.labeling import PandasLFApplier, LFAnalysis

lfs = [transe_sim, rule1_transe_sim, rule2_transe_sim,rule3_transe_sim,rule4_transe_sim, rule5_transe_sim, rule6_transe_sim,rule7_transe_sim,rule8_transe_sim,rule9_transe_sim, rule10_transe_sim]


applier = PandasLFApplier(lfs=lfs)
L_train = applier.apply(df=df_train)
from snorkel.labeling.model import LabelModel

label_model = LabelModel(cardinality=2, verbose=True)
label_model.fit(L_train=L_train, n_epochs=500, log_freq=100, seed=123)
preds_train = label_model.predict(L=L_train)
probs_train =label_model.predict_proba(L=L_train)
probs_train[:, 1]
sum(probs_train[:, 1] > 0.5)
df_train['prob'] = probs_train[:, 1]
df_het_test['label'] =1
from snorkel.labeling import PandasLFApplier, LFAnalysis
applier = PandasLFApplier(lfs)
L_dev = applier.apply(df_het_test)
LFAnalysis(L_dev, lfs).lf_summary(df_het_test.label.values)
df_rank = {}
df_train.index = df_train.DB_ID
for i,drug in enumerate(df_train.DB_ID.unique()):
    x =  df_train[df_train.DB_ID==drug]
    x['Rank'] = x['prob'].rank(method='first',ascending=False)
    x.index = x.DO_ID
    df_rank[drug]= x['Rank'].to_dict()
    
pos_pair_dict = {}
for i,row in df_het_test.iterrows():
    if row['DB_ID'] not in pos_pair_dict:
        pos_pair_dict[row['DB_ID']] =[]
    pos_pair_dict[row['DB_ID']].append( row['DO_ID'])
list(pos_pair_dict.items())[:10]
def mrr_map_new(pos_pair_dict, df_rank):
    rank_scores = []
    ave_p = []
    in_top_5 = 0
    in_top_10 = 0
    print('number of positives: ',len(pos_pair_dict.keys()))

    for fromt in pos_pair_dict.keys():
        relevant_ranks = []
        min_rank = 9999999999999999
        for asdf in pos_pair_dict[fromt]:
            rank = df_rank[fromt][asdf]
            relevant_ranks.append(rank)
            if rank < min_rank:
                min_rank = rank
        rank = min_rank

        rank_scores.append(1.0/rank)
        if min_rank <= 5:
            in_top_5 += 1
        if min_rank <= 10:
            in_top_10 += 1

        precisions = []
        for rank in relevant_ranks:
            good_docs = len([r for r in relevant_ranks if r <= rank])
            precisions.append(good_docs/rank)
        if len(precisions) == 0:
            precisions = [0]
        ave_p.append(np.mean(precisions))

    print('mean average precision: ', np.mean(ave_p))
    print('mean reciprocal rank: ', np.mean(rank_scores))
    print('recallrate at 5: ', in_top_5/len(pos_pair_dict.keys()))
    print('recallrate at 10: ', in_top_10/len(pos_pair_dict.keys()))
    print('formatted')
    print(round(np.mean(ave_p), 3),' & ',round(np.mean(rank_scores), 3),' & ',round(in_top_5/len(pos_pair_dict.keys()), 3),' & ',round(in_top_10/len(pos_pair_dict.keys()), 3))
    return round(np.mean(ave_p), 3)
mrr_map_new(pos_pair_dict, df_rank)<jupyter_output>number of positives:  117
mean average precision:  0.3447415668897847
mean reciprocal rank:  0.34761288094621434
recallrate at 5:  0.6068376068376068
recallrate at 10:  0.9572649572649573
formatted
0.345  &  0.348  &  0.607  &  0.957
{'fraction_non_alphanumeric': 0.12010981468771448, 'fraction_numerical': 0.023599598754025657, 'mean_word_length': 3.7366841710427607, 'pattern_counts': {'":': 0, '<': 67, '<?xml version=': 0, '>': 69, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '497386', 'license': 'MIT', 'n_tokens_mistral': 7101, 'n_tokens_neox': 6688, 'n_words': 1396, 'path': 'notebook/Probablistic Rules-Snorkel-DrugRepurposing.ipynb', 'repo_name': 'MaastrichtU-IDS/KGRulEm', 'size': '18941'}