<jupyter_start><jupyter_text>Build a medium size KG from a CSV dataset First let's initialize the KG object as we did previously:<jupyter_code>import kglab

namespaces = {
    "wtm":  "http://purl.org/heals/food/",
    "ind":  "http://purl.org/heals/ingredient/",
    "skos": "http://www.w3.org/2004/02/skos/core#",
    }

kg = kglab.KnowledgeGraph(
    name = "A recipe KG example based on Food.com",
    base_uri = "https://www.food.com/recipe/",
    namespaces = namespaces,
    )<jupyter_output><empty_output><jupyter_text>Here's a way to describe the namespaces that are available to use:<jupyter_code>kg.describe_ns()<jupyter_output><empty_output><jupyter_text>Next, we'll define a dictionary that maps (somewhat magically) from strings (i.e., "labels") to ingredients defined in the  vocabulary:<jupyter_code>common_ingredient = {
    "water": kg.get_ns("ind").Water,
    "salt": kg.get_ns("ind").Salt,
    "pepper": kg.get_ns("ind").BlackPepper,
    "black pepper": kg.get_ns("ind").BlackPepper,
    "dried basil": kg.get_ns("ind").Basil,

    "butter": kg.get_ns("ind").Butter,
    "milk": kg.get_ns("ind").CowMilk,
    "egg": kg.get_ns("ind").ChickenEgg,
    "eggs": kg.get_ns("ind").ChickenEgg,
    "bacon": kg.get_ns("ind").Bacon,

    "sugar": kg.get_ns("ind").WhiteSugar,
    "brown sugar": kg.get_ns("ind").BrownSugar,
    "honey": kg.get_ns("ind").Honey,
    "vanilla": kg.get_ns("ind").VanillaExtract,
    "vanilla extract": kg.get_ns("ind").VanillaExtract,

    "flour": kg.get_ns("ind").AllPurposeFlour,
    "all-purpose flour": kg.get_ns("ind").AllPurposeFlour,
    "whole wheat flour": kg.get_ns("ind").WholeWheatFlour,

    "olive oil": kg.get_ns("ind").OliveOil,
    "vinegar": kg.get_ns("ind").AppleCiderVinegar,

    "garlic": kg.get_ns("ind").Garlic,
    "garlic clove": kg.get_ns("ind").Garlic,
    "garlic cloves": kg.get_ns("ind").Garlic,

    "onion": kg.get_ns("ind").Onion,
    "onions": kg.get_ns("ind").Onion,
    "cabbage": kg.get_ns("ind").Cabbage,
    "carrot": kg.get_ns("ind").Carrot,
    "carrots": kg.get_ns("ind").Carrot,
    "celery": kg.get_ns("ind").Celery,
    "potato": kg.get_ns("ind").Potato,
    "potatoes": kg.get_ns("ind").Potato,
    "tomato": kg.get_ns("ind").Tomato,
    "tomatoes": kg.get_ns("ind").Tomato,
    
    "baking powder": kg.get_ns("ind").BakingPowder,
    "baking soda": kg.get_ns("ind").BakingSoda,
}<jupyter_output><empty_output><jupyter_text>This is where use of NLP work to produce *annotations* begins to overlap with KG pratices. Now let's load our dataset of recipes – the `dat/recipes.csv` file in CSV format – into a `pandas` dataframe:<jupyter_code>from os.path import dirname
import os
import pandas as pd

df = pd.read_csv(dirname(os.getcwd()) + "/dat/recipes.csv")
df.head()<jupyter_output><empty_output><jupyter_text>Then iterate over the rows in the dataframe, representing a recipe in the KG for each row:<jupyter_code>import rdflib

for index, row in df.iterrows():
    recipe_id = row["id"]
    node = rdflib.URIRef("https://www.food.com/recipe/{}".format(recipe_id))
    kg.add(node, kg.get_ns("rdf").type, kg.get_ns("wtm").Recipe)

    recipe_name = row["name"]
    kg.add(node, kg.get_ns("skos").definition, rdflib.Literal(recipe_name))
    
    cook_time = row["minutes"]
    cook_time_literal = "PT{}M".format(int(cook_time))
    code_time_node = rdflib.Literal(cook_time_literal, datatype=kg.get_ns("xsd").duration)
    kg.add(node, kg.get_ns("wtm").hasCookTime, code_time_node)
    
    ind_list = eval(row["ingredients"])

    for ind in ind_list:
        ingredient = ind.strip()
        ingredient_obj = common_ingredient[ingredient]
        kg.add(node, kg.get_ns("wtm").hasIngredient, ingredient_obj)<jupyter_output><empty_output><jupyter_text>Notice how the `xsd:duration` literal is now getting used to represent cooking times.We've structured this example such that each of the recipes in the CSV file has a known representation for all of its ingredients.There are nearly 250K recipes in the full dataset from  so the `common_ingredient` dictionary would need to be extended quite a lot to handle all of those possible ingredients. At this stage, our graph has grown by a couple orders of magnitude, so its visualization should be more interesting now.Let's take a look:<jupyter_code>VIS_STYLE = {
    "wtm": {
        "color": "orange",
        "size": 20,
    },
    "ind":{
        "color": "blue",
        "size": 35,
    },
}

subgraph = kglab.SubgraphTensor(kg)
pyvis_graph = subgraph.build_pyvis_graph(notebook=True, style=VIS_STYLE)

pyvis_graph.force_atlas_2based()
pyvis_graph.show("tmp.fig01.html")<jupyter_output><empty_output><jupyter_text>Given the defaults for this kind of visualization, there's likely a dense center mass of orange (recipes) at the center, with a close cluster of common ingredients (dark blue), surrounded by less common ingredients and cooking times (light blue).  Performance analysis of serialization methods Let's serialize this recipe KG constructed from the CSV dataset to a local TTL file, while measuring the time and disk space required:<jupyter_code>import time

write_times = []

t0 = time.time()
kg.save_rdf("tmp.ttl")
write_times.append(round((time.time() - t0) * 1000.0, 2))<jupyter_output><empty_output><jupyter_text>Let's also serialize the KG into the other formats that we've been using, to compare relative sizes for a medium size KG:<jupyter_code>t0 = time.time()
kg.save_rdf("tmp.xml", format="xml")
write_times.append(round((time.time() - t0) * 1000.0, 2))

t0 = time.time()
kg.save_jsonld("tmp.jsonld")
write_times.append(round((time.time() - t0) * 1000.0, 2))

t0 = time.time()
kg.save_parquet("tmp.parquet")
write_times.append(round((time.time() - t0) * 1000.0, 2))
file_paths = ["tmp.ttl", "tmp.xml", "tmp.jsonld", "tmp.parquet"]
file_sizes = [os.path.getsize(file_path) for file_path in file_paths]

df = pd.DataFrame({"file_path": file_paths, "file_size": file_sizes, "write_time": write_times})
df["ms_per_byte"] = df["write_time"] / df["file_size"]
df<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15128968253968253, 'fraction_numerical': 0.008101851851851851, 'mean_word_length': 4.695856873822976, 'pattern_counts': {'":': 47, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 2, 'lorem ipsum': 0, 'www.': 3, 'xml': 3}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '469965', 'license': 'MIT', 'n_tokens_mistral': 2277, 'n_tokens_neox': 2178, 'n_words': 500, 'path': 'examples/ex2_0.ipynb', 'repo_name': 'ArenasGuerreroJulian/kglab', 'size': '6048'}