<jupyter_start><jupyter_text>Knowledge Graph Embedding using the Wikimedia knowledge graph data.In this notebook, we will show using the Wikimedia knowledge graph data, how you can compute embeddings for your knowledge graph and perform downstream task completion such as node classification and link prediction.  Data PreparationFirst, you can download the data from kaggle using this link [here](https://www.kaggle.com/kenshoresearch/kensho-derived-wikimedia-data). The zip file is 24GB in total. For our example, we only need the property.csv, item.csv and statements.csv. Once you have downloaded the file, you can unzip the files in the ./data/wikimedia/ folder.<jupyter_code>!unzip file.zip -d ./data/wikimedia
!conda install -c dglteam dgl-cuda10.1 -y
!pip install rdflib
!pip install dglke
!python3 -m spacy download en_core_web_lg
%load_ext autoreload
%autoreload 2

%matplotlib notebook
import pandas as pd


import numpy as np
import pylab as plt
import seaborn as sns
sns.set_style('ticks')

import seaborn as sns
import os
import json
import boto3<jupyter_output><empty_output><jupyter_text>Let's load the relation dictionary file.<jupyter_code>dfRelations = pd.read_csv("./data/wikimedia/property.csv")
dfRelations.head()
#Total Counts
dfRelations.count()<jupyter_output><empty_output><jupyter_text>Let's find out if we have any zeros. Looks like some properties do not have descriptions.<jupyter_code>dfRelations.isnull().sum()<jupyter_output><empty_output><jupyter_text>Let's load the entity file.<jupyter_code>dfEntities = pd.read_csv("./data/wikimedia/item.csv")
dfEntities.head()
#Total Counts
dfEntities.count()
#Null Counts
dfEntities.isnull().sum()
#Drop description column.
dfEntitiesClean = dfEntities.drop('en_description', 1)
#drop any null values.
dfEntitiesClean = dfEntities.dropna()
dfEntitiesClean.count()<jupyter_output><empty_output><jupyter_text>Let's load the statements.<jupyter_code>dfTriple = pd.read_csv("./data/wikimedia/statements.csv")
dfTriple.head()<jupyter_output><empty_output><jupyter_text>Let's join the properties and the statements file.<jupyter_code>dfResults=pd.merge(dfTriple, dfRelations, left_on='edge_property_id', right_on='property_id')


dfResults.head()<jupyter_output><empty_output><jupyter_text>Let's join the entity and the head columns.<jupyter_code>dfResultsHR=pd.merge(dfResults, dfEntities, left_on='source_item_id', right_on='item_id')
dfResultsHR.head()<jupyter_output><empty_output><jupyter_text>Let's join the entity and the tail columns.<jupyter_code>dfResultsHRT=pd.merge(dfResultsHR, dfEntities,how='left', left_on='target_item_id', right_on='item_id')
dfResultsHRT.head()<jupyter_output><empty_output><jupyter_text>Let's select and rename the head, relation and tail to [h,r,t] columns.<jupyter_code>dfResultsFinal = dfResultsHRT[['en_label_y','en_label_x', 'en_label']].rename(columns={"en_label_y": "head", "en_label_x": "relation", "en_label":"tail"})
dfResultsFinal.head()
#Count null values
dfResultsFinal.isnull().sum()
#Drop null records.
dfTrain = dfResultsFinal.dropna()<jupyter_output><empty_output><jupyter_text>Our Final dataset contains about 120 mio triples or statements.<jupyter_code>dfTrain.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 119976844 entries, 0 to 141206852
Data columns (total 3 columns):
head        object
relation    object
tail        object
dtypes: object(3)
memory usage: 3.6+ GB<jupyter_text>Let's split our data into training, test and valid datasets.<jupyter_code>from sklearn.model_selection import train_test_split

trainDF, testDF = train_test_split(dfTrain, test_size=0.1)

trainDF, validDF = train_test_split(trainDF, test_size=0.15)
trainDF.shape, validDF.shape, testDF.shape<jupyter_output><empty_output><jupyter_text>Let's save the our files as tabular separated files.<jupyter_code>trainDF.to_csv("./data/wikimedia/train.txt", header = None, index = None, sep = "\t")
validDF.to_csv("./data/wikimedia/valid.txt", header = None, index = None, sep = "\t")
testDF.to_csv("./data/wikimedia/test.txt", header = None, index = None, sep = "\t")<jupyter_output><empty_output><jupyter_text>Knowledge Graph Embeddings  Let's run the command line to generage our embeddings. In our case we will mix cpu-gpu as our KG is quite large.<jupyter_code>%%time
!DGLBACKEND=pytorch dglke_train --dataset wikimedia --model_name TransE_l2 --batch_size 1000 \
--neg_sample_size 200 --hidden_dim 400 --gamma 19.9 --lr 0.25 --max_step 24000 --log_interval 100 \
--batch_size_eval 16 -adv --regularization_coef 1.00E-09 --test --gpu 0 1 2 3 --mix_cpu_gpu  --save_path ./wikimedia \
--data_path ./data/wikimedia/ --format raw_udd_hrt --data_files train.txt valid.txt test.txt --neg_sample_size_eval 10000<jupyter_output><empty_output><jupyter_text>Let's evaluate our model.<jupyter_code>%%time
!DGLBACKEND=pytorch dglke_eval --dataset wikimedia --model_name TransE_l2 \
--neg_sample_size 200 --hidden_dim 400 --gamma 19.9 \
--batch_size_eval 16 --gpu 0 1 2 3  --model_path ./wikimedia/TransE_l2_wikimedia_0/ \
--data_path ./data/wikimedia/ --format raw_udd_hrt --data_files train.txt valid.txt test.txt --neg_sample_size_eval 10000 --no_eval_filter<jupyter_output>Using backend: pytorch
Reading train triples....
Finished. Read 91782285 train triples.
Reading valid triples....
Finished. Read 16196874 valid triples.
Reading test triples....
Finished. Read 11997685 test triples.
Logs are being recorded at: ./wikimedia/TransE_l2_wikimedia_0/eval.log
/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/dgl/base.py:25: UserWarning: multigraph will be deprecated.DGL will treat all graphs as multigraph in the future.
  warnings.warn(msg, warn_type)
|valid|: 16196874
|test|: 11997685
-------------- Test result --------------
Test average MRR: 0.4159753346227368
Test average MR: 1001.1689418833716
Test average HITS@1: 0.3540242971873324
Test average HITS@3: 0.45541123141672746
Test average HITS@10: 0.5213350742247359
-----------------------------------------
Test takes 6781.685 seconds
CPU times: user 2min 7s, sys: 30 s, total: 2min 37s
Wall time: 2h 9min 59s<jupyter_text>Let's run the inference command line. In our case we want to find the Top 5 similar nodes to our "head.list" file. Our file contains the following entries:<NAME><jupyter_code># Using PyTorch Backend
!DGLBACKEND=pytorch dglke_score --data_path ./data/wikimedia/ --model_path ./wikimedia/TransE_l2_wikimedia_0/ \
--format 'h_*_*' --data_files head.list \
--score_func none --topK 5<jupyter_output>Traceback (most recent call last):
  File "/usr/local/bin/dglke_score", line 11, in <module>
    load_entry_point('dglke==0.1.0.dev0', 'console_scripts', 'dglke_score')()
  File "/usr/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.6/site-packages/dglke-0.1.0.dev0-py3.6.egg/dglke/infer_score.py", line 25, in <module>
  File "/usr/local/lib/python3.6/site-packages/dglke-0.1.0.dev0-py3.6.egg/dglke/models/__init__.py", line 20, in <module>
  File "/usr/local/lib/pyt[...]
{'fraction_non_alphanumeric': 0.125, 'fraction_numerical': 0.04228723404255319, 'mean_word_length': 5.304274937133277, 'pattern_counts': {'":': 3, '<': 63, '<?xml version=': 0, '>': 63, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '129320', 'license': 'Apache-2.0', 'n_tokens_mistral': 2830, 'n_tokens_neox': 2596, 'n_words': 643, 'path': 'sagemaker-python-sdk/dgl_kge/kge_wikimedia.ipynb', 'repo_name': 'phiamazon/amazon-sagemaker-examples', 'size': '7536'}