<jupyter_start><jupyter_text>Statistics of DatasetIn this part we want to know how many nodes and relations our graph has Output:  relation and  nodes<jupyter_code>Relation2id = {j:i for i,j in enumerate(train.keys()[2:])}
Id2Relation = {j:i for i , j in Relation2id.items()}
Entities = list(set(train['source_name'].values.tolist() + train['destination_name'].values.tolist() + test['source_name'].values.tolist() + test['destination_name'].values.tolist()))
Entities2id = {j:i for i,j in enumerate(Entities)}
len(Relation2id),len(Entities)<jupyter_output><empty_output><jupyter_text>Dataset to triples: paris france 01000000 => paris capitalof parisFor using some embedding libaries such as OpenKE we need to transfor our dataset to triples : (head, tail, relation)<jupyter_code>def csv2triple(train):
  Train_triples = []
  for row in train.values.tolist():
    head = row[0]
    tail = row[1]
    for i in range(len(row[2:])):
      if row[2+i] == 1:
        relation = Id2Relation[i]
        Train_triples.append([head,tail,relation])
  return Train_triples

Train_triples = csv2triple(train)
Test_triples  = csv2triple(test)
len(Train_triples),len(Test_triples)
Train_triples[:10]<jupyter_output><empty_output><jupyter_text>Now we should convert Dataset to numerics<jupyter_code>def Triples2IDs(triples):
  for i in range(len(triples)):
    triples[i][0] = Entities2id[triples[i][0]]
    triples[i][1] = Entities2id[triples[i][1]]
    triples[i][2] = Relation2id[triples[i][2]]
  return triples    
train = Triples2IDs(Train_triples)
test  = Triples2IDs(Test_triples)<jupyter_output><empty_output><jupyter_text>Conver to files<jupyter_code>train[:5]
def graph2file(triples,file_name):
  file = open('OpenKE format/' + str(file_name),'w')
  file.write(str(len(triples))+ '\n')
  for i in range(len(triples)):
    file.write(str(triples[i][0]) +' '+ str(triples[i][1]) +' '+ str(triples[i][2])+'\n')
  file.close()
graph2file(train,'train2id.txt')
graph2file(test,'test2id.txt')
graph2file(test,'valid2id.txt') # the same BC we need it
file = open('OpenKE format/relation2id.txt','w')
file.write(str(len(Id2Relation))+ '\n')
for k,v in Id2Relation.items():
  file.write(str(k) +'\t'+ str(v)+'\n')
file.close()
file = open('OpenKE format/entity2id.txt','w')
file.write(str(len(Entities2id))+ '\n')
for k,v in Entities2id.items():
  file.write(str(k) +'\t'+ str(v)+'\n')
file.close()
!zip -r 'opke.zip' 'OpenKE format'<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1653061224489796, 'fraction_numerical': 0.022040816326530613, 'mean_word_length': 6.316417910447761, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '391639', 'license': 'MIT', 'n_tokens_mistral': 950, 'n_tokens_neox': 932, 'n_words': 180, 'path': 'resources/notebooks/RezoJDM_Dataset2Triples.ipynb', 'repo_name': 'ContentSide/rezojdm-sds', 'size': '2450'}