<jupyter_start><jupyter_text>Hatchet Query LanguageThis notebook explores [Hatchet](https://github.com/LLNL/hatchet) queries from its [**Object-based Dialect**](https://hatchet.readthedocs.io/en/latest/query_lang.html), specifically **Category 1: Quantifier Capabilities**. The notebook covers different query types that are capable of matching various number of nodes from [GraphFrame](https://hatchet.readthedocs.io/en/latest/user_guide.html) objects.<jupyter_code># display documentation for object-based dialect
from IPython.display import Markdown, display
display(Markdown("../common/documentation/object-based-dialect-01.md"))
# display dataset information 
display(Markdown("../common/documentation/dataset-information.md"))
import hatchet as ht
gf = ht.GraphFrame.from_caliper("../../data/lulesh-16nodes/lulesh-annotation-profile-512cores.json")
# display GraphFrame information 
display(Markdown("../common/documentation/graph-tree-information.md"))
print(gf.tree())
# display DataFrame information 
display(Markdown("../common/documentation/dataframe-information.md"))
gf.dataframe
# why use drop index levels
display(Markdown("../common/documentation/drop-index-information.md"))
gf.drop_index_levels()
# display query type 1 documentation
display(Markdown("../common/documentation/quantifier-capabilities-01.md"))
# single node with names starting with MPI_Barrier
query_1 = [
    (".", {"name": "MPI_Barrier.*"}),
]<jupyter_output><empty_output><jupyter_text>The above query is passed to Hatchet’s `filter()` function to filter the input GraphFrame. The `filter()` function takes a user-supplied function or query object and applies that to all rows in the DataFrame. The resulting Series or DataFrame is used to filter the DataFrame to only return rows that are true.<jupyter_code>gf_filt = gf.filter(query_1)<jupyter_output><empty_output><jupyter_text>The resulting GraphFrame now only lists the  nodes that matched the query:<jupyter_code>print(gf_filt.tree())<jupyter_output><empty_output><jupyter_text>The query match is also reflected in the DataFrame:<jupyter_code>gf_filt.dataframe
# display query type 2 documentation
display(Markdown("../common/documentation/quantifier-capabilities-02-01.md"))
# zero or more nodes with name starting with Calc
query_2_1 = [
    ("*", {"name": "Calc.*"}),
]<jupyter_output><empty_output><jupyter_text>Just as before, the above query is passed to Hatchet’s `filter()` function to filter the input GraphFrame.<jupyter_code>gf_filt = gf.filter(query_2_1)<jupyter_output><empty_output><jupyter_text>Here, instead of matching only single nodes, entire call stacks can be matched. The resulting GraphFrame now only lists the  nodes that matched the query:<jupyter_code>print(gf_filt.tree())<jupyter_output><empty_output><jupyter_text>The query match is also reflected in the DataFrame:<jupyter_code>gf_filt.dataframe
# display query type 2 documentation
display(Markdown("../common/documentation/quantifier-capabilities-02-02.md"))
# zero or more nodes with several query nodes
query_2_2 = [
    (".", {"name": "lulesh.*"}),
    ("."),
    ("*", {"name": "Calc.*"})
]<jupyter_output><empty_output><jupyter_text>Just as before, the above query is passed to Hatchet’s `filter()` function to filter the input GraphFrame.<jupyter_code>gf_filt = gf.filter(query_2_2)<jupyter_output><empty_output><jupyter_text>The resulting GraphFrame now only lists the  nodes that matched the query:<jupyter_code>print(gf_filt.tree())
# display query type 2 documentation
display(Markdown("../common/documentation/quantifier-capabilities-02-03.md"))
gf_filt.dataframe
# display query type 3 documentation
display(Markdown("../common/documentation/quantifier-capabilities-03-01.md"))
# one or more nodes with name starting with CalcMonotonic
query_3_1 = [
    ("+", {"name": "CalcMonotonic.*"}),
]<jupyter_output><empty_output><jupyter_text>Just as before, the above query is passed to Hatchet’s `filter()` function to filter the input GraphFrame.<jupyter_code>gf_filt = gf.filter(query_3_1)<jupyter_output><empty_output><jupyter_text>The resulting GraphFrame now only lists the  nodes that matched the query:<jupyter_code>print(gf_filt.tree())<jupyter_output><empty_output><jupyter_text>The query match is also reflected in the DataFrame:<jupyter_code>gf_filt.dataframe
# display query type 3 documentation
display(Markdown("../common/documentation/quantifier-capabilities-03-02.md"))
# one or more nodes with several query nodes
query_3_2 = [
    (".", {"name": "lulesh.*"}),
    ("."),
    ("+", {"name": "Calc.*"})
]<jupyter_output><empty_output><jupyter_text>Just as before, the above query is passed to Hatchet’s `filter()` function to filter the input GraphFrame.<jupyter_code>gf_filt = gf.filter(query_3_2)<jupyter_output><empty_output><jupyter_text>The resulting GraphFrame now only lists the  nodes that matched the query:<jupyter_code>print(gf_filt.tree())
# display query type 3 documentation
display(Markdown("../common/documentation/quantifier-capabilities-03-03.md"))
gf_filt.dataframe
# display query type 4 documentation
display(Markdown("../common/documentation/quantifier-capabilities-04.md"))
# exactly three nodes with names starting with Calc
query_4 = [
    (3, {"name": "Calc.*"}),
]<jupyter_output><empty_output><jupyter_text>Just as before, the above query is passed to Hatchet’s `filter()` function to filter the input GraphFrame.<jupyter_code>gf_filt = gf.filter(query_4)<jupyter_output><empty_output><jupyter_text>The resulting GraphFrame now only lists the  nodes that matched the query:<jupyter_code>print(gf_filt.tree())<jupyter_output><empty_output><jupyter_text>The query match is also reflected in the DataFrame:<jupyter_code>gf_filt.dataframe<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12179822268687925, 'fraction_numerical': 0.011326014985189058, 'mean_word_length': 6.798913043478261, 'pattern_counts': {'":': 8, '<': 69, '<?xml version=': 0, '>': 69, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '266154', 'license': 'MIT', 'n_tokens_mistral': 1879, 'n_tokens_neox': 1782, 'n_words': 447, 'path': 'notebooks/query-language-object-based-dialect/01.01.quantifier_capabilities.ipynb', 'repo_name': 'LLNL/hatchet-tutorial', 'size': '5739'}