<jupyter_start><jupyter_text>Running SPARQL queries[SPARQL](https://www.w3.org/TR/rdf-sparql-query/) is a standard query language for RDF.Note: there are also other approaches for queries, such [SHACL](https://www.w3.org/TR/shacl/) the *Shapes Constraint Language*.In any case, if you've worked with SQL queries before, SPARQL should seem quite familiar.OH: >"If you build it, they will SPARQL" First, let's define a small graph in RDF:<jupyter_code>n3_string = """
@prefix : <http://www.w3.org/2012/12/rdf-val/SOTA-ex#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/'> .

:peep0 a foaf:Person ;
    foaf:givenName "Alice" ;
    foaf:familyName "Nakamoto" ;
    foaf:phone <tel:+1.555.222.2222> ;
    foaf:mbox <mailto:<EMAIL>> .

:peep1 a foaf:Person ;
    foaf:givenName "Bob" ;
    foaf:familyName "Patel" ;
    foaf:phone <tel:+1.555.666.5150> ;
    foaf:mbox <mailto:<EMAIL>> .

:peep2 a foaf:Person ;
    foaf:givenName "Dhanya" ;
    foaf:familyName "O'Neill" ;
    foaf:phone <tel:+1.555.123.9876> ;
    foaf:mbox <mailto:<EMAIL>> .
"""<jupyter_output><empty_output><jupyter_text>Then construct a graph using `rdflib`:<jupyter_code>import rdflib as rdf
import rdflib.namespace

g = rdf.Graph()
g.parse(data=n3_string, format="n3")<jupyter_output><empty_output><jupyter_text>Now we'll define a SPARQL query to extract the surnames and email addresses for each person in the graph, in descending order by surname:<jupyter_code>query = """
    SELECT ?person ?surname ?email
    WHERE {
        ?person foaf:familyName ?surname .
        ?person foaf:mbox ?email .
    }
    ORDER BY DESC(?surname)
"""<jupyter_output><empty_output><jupyter_text>Then run the SPARQL query and iterate over its result set:<jupyter_code>for row in g.query(query):
    print(row)<jupyter_output>(rdflib.term.URIRef('http://www.w3.org/2012/12/rdf-val/SOTA-ex#peep1'), rdflib.term.Literal('Patel'), rdflib.term.URIRef('mailto:<EMAIL>'))
(rdflib.term.URIRef('http://www.w3.org/2012/12/rdf-val/SOTA-ex#peep2'), rdflib.term.Literal("O'Neill"), rdflib.term.URIRef('mailto:<EMAIL>'))
(rdflib.term.URIRef('http://www.w3.org/2012/12/rdf-val/SOTA-ex#peep0'), rdflib.term.Literal('Nakamoto'), rdflib.term.URIRef('mailto:<EMAIL>'))<jupyter_text>To access the URLs and literals from those results in more readable form, use the `n3()` function with the graph's namespaces to extract formatted text from each term:<jupyter_code>for row in g.query(query):
    person = row[0].n3(g.namespace_manager)
    surname = eval(row[1].n3())
    email = row[2].n3()

    print(person, surname, email)<jupyter_output>:peep1 Patel <mailto:<EMAIL>>
:peep2 O'Neill <mailto:<EMAIL>>
:peep0 Nakamoto <mailto:<EMAIL>><jupyter_text>--- SPARQL queries with `kglab` abstractions Now we'll load our medium size KG serialized as `tmp.ttl` from the previous example – although note that we're adding `nom:` as an additional vocabulary namespace:<jupyter_code>import kglab

namespaces = {
    "wtm": "http://purl.org/heals/food/",
    "ind": "http://purl.org/heals/ingredient/",
    "nom": "https://github.com/DerwenAI/kglab/wiki/Vocab#",
    }

kg = kglab.KnowledgeGraph(
    name = "A recipe KG example based on Food.com",
    base_uri = "https://www.food.com/recipe/",
    language = "en",
    namespaces = namespaces,
    )

kg.load_ttl("tmp.ttl")<jupyter_output><empty_output><jupyter_text>Identify recipes with special ingredients and cooking times Let's define a query based on [SPARQL 1.1](https://www.w3.org/TR/sparql11-query/) to look for recipes that use *eggs* and require at least 1.5 hours to prepare:<jupyter_code>query = """
SELECT ?rec ?def ?time
  WHERE {
      ?rec rdf:type wtm:Recipe .
      ?rec skos:definition ?def .
      ?rec wtm:hasIngredient ind:ChickenEgg .
      ?rec wtm:hasCookTime ?time .
      FILTER (?time >= 90)
  }
  ORDER BY DESC(?time)
        """<jupyter_output><empty_output><jupyter_text>Now we'll run the query and iterate through its result set:<jupyter_code>for row in kg.query(query):
    rec, recipe_name, cook_time = row
    print("{}\n {}\n {} minutes\n".format(rec, recipe_name, cook_time))<jupyter_output>https://www.food.com/recipe/31041
 marti s homemade noodles
 160 minutes

https://www.food.com/recipe/124106
 hungarian noodles homemade
 120 minutes

https://www.food.com/recipe/371915
 crepes    alton brown
 100 minutes

https://www.food.com/recipe/77354
 wonton wrappers
 90 minutes

https://www.food.com/recipe/19104
 all purpose dinner crepes batter
 90 minutes

https://www.food.com/recipe/497918
 easy homemade noodles
 90 minutes<jupyter_text>That's interesting – notice the outliers based on cooking times, i.e., recipes for special kinds of *noodles*. Some of these take 2-3 hours to prepare.  Using SPARQL queries and post-processing to create annotationsLet's begin to add [*annotations*](https://www.w3.org/TR/annotation-model/) to the KG.In other words, we'll add descriptions to the recipes to help classify them based on an [*ontology*](https://www.datasciencecentral.com/profiles/blogs/understanding-how-taxonomies-differ-from-ontologies).We've created a brief *controlled vocabulary* in RDF called `NOM` in the `nom.ttl` file, which provides part of a simple taxonomy for our recipes.Take a look at those definitions – mostly these are distinctions between two kind of flour-based recipes:  * "pancakes" -- which use batter in their preparation  * "noodles" -- which use kneading in their preparation  Let's load these definitions into our KG:<jupyter_code>kg.load_ttl("nom.ttl")<jupyter_output><empty_output><jupyter_text>Now we'll query the KG, specifically looking for recipes that use *flour*, *eggs*, *salt* among their ingredients:<jupyter_code>query = """
SELECT ?rec ?def
  WHERE {
    ?rec rdf:type wtm:Recipe .
    ?rec skos:definition ?def .
    ?rec wtm:hasIngredient ind:ChickenEgg .
    ?rec wtm:hasIngredient ind:AllPurposeFlour .
    ?rec wtm:hasIngredient ind:Salt
  }
        """<jupyter_output><empty_output><jupyter_text>Next, while looping through the query result set, we'll perform some post-processing which would be quite difficult to express in the query language, although relatively simple within a Python loop.We can compare two points for each recipe in the results:  * Is the word noodle or its close synonyms in the recipe description?  * Does the recipe use butter?  Our `nom.ttl` ontology loaded above distinguishes between a `nom:Noodle` and a `nom:Pancake` as two categories of flour-based recipes. Let's run this look and review the results, then determine whether we might be able to specific rules to distinguish between noodle-ish foods and pancake-ish foods.  For example, using butter is rare among noodle recipes that also use eggs, though more common for pancakes.<jupyter_code>tally = []

for row in kg.query(query):
    url, recipe_name = row
    
    # collect two flags to evaluate our noodle vs. pancake rules
    has_butter = (url, kg.get_ns("wtm").hasIngredient, kg.get_ns("ind").Butter) in kg._g
    sez_noodle =  any([x in recipe_name for x in ["noodle", "spaetzle", "dumpling", "pasta"]])
    
    tally.append(str([ has_butter, sez_noodle ]))
    print("{}\n {}\n {} {}\n".format(url, recipe_name, has_butter, sez_noodle))<jupyter_output>https://www.food.com/recipe/277824
 easy basic crepes
 False False

https://www.food.com/recipe/317697
 swabian noodles  spatzle
 False True

https://www.food.com/recipe/441475
 swiss spaetzle  very easy homemade noodle
 False True

https://www.food.com/recipe/60413
 never fail popovers
 False False

https://www.food.com/recipe/111008
 flop proof crepes
 True False

https://www.food.com/recipe/137357
 dane dumplings
 True True

https://www.food.com/recipe/72129
 muffin tin popovers
 False False

https://www.food.com/recipe/262038
 crepe batter
 True False

https://www.food.com/recipe/497918
 easy homemade noodles
 False True

https://www.food.com/recipe/140570
 rivilchas
 False False

https://www.food.com/recipe/19104
 all purpose dinner crepes batter
 True False

https://www.food.com/recipe/186892
 pate a choux
 True False

https://www.food.com/recipe/157638
 hootenanny pancakes
 True False

https://www.food.com/recipe/280663
 yorkhire pudding aka popovers
 False False

https://www.fo[...]<jupyter_text>Let's tally a *confusion matrix* based on what we observed from the query results:<jupyter_code>from collections import Counter

Counter(tally)<jupyter_output><empty_output><jupyter_text>In this sample the conditional probability `P(sez_noodle|has_butter)` is approximately 1:2, while `P(has_butter|sez_noodle)` is approximately 1:4 (i.e., significantly lower).So these conditions seem reasonably good to use as filters for identifying which recipes to annotate.Of course, not every recipe is either a kind of pancake or noodle, since there are recipes for *omelette*, *Yorkshire pudding*, etc. We'll revisit this kind of problem later with more formal, semi-automated methods that leverage machine learning.For now let's add annotations based on our manual review:<jupyter_code>noodle_ids = { "400", "86710", "331765", "508734", "320154", "220361", "148900", "317697", "252783", "137357", "1975", "31041", "441475", "261361", "124106", "78459", "358908", "103964", "91311", "497918", "328388" }
pancake_ids = { "277824", "489", "4643", "272746", "12055", "124131", "40772", "459", "48178", "124176", "61108", "111008", "262038", "458", "440398", "157638" }
import rdflib as rdf
from rdflib.namespace import RDF, XSD

for id in noodle_ids:
    url = f"https://www.food.com/recipe/{id}"
    node = rdf.URIRef(url)
    kg.add(node, RDF.type, kg.get_ns("nom").Noodle)

for id in pancake_ids:
    url = f"https://www.food.com/recipe/{id}"
    node = rdf.URIRef(url)
    kg.add(node, RDF.type, kg.get_ns("nom").Pancake)

kg.save_ttl("tmp.ttl")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12767911818738517, 'fraction_numerical': 0.04358032251479894, 'mean_word_length': 4.5361581920903955, 'pattern_counts': {'":': 3, '<': 69, '<?xml version=': 0, '>': 71, 'https://': 30, 'lorem ipsum': 0, 'www.': 33, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '18772', 'license': 'MIT', 'n_tokens_mistral': 3797, 'n_tokens_neox': 3427, 'n_words': 943, 'path': 'ex01_5.ipynb', 'repo_name': 'dmoore247/kglab', 'size': '9885'}