<jupyter_start><jupyter_text>We are using NitroML on Kubeflow: This notebook allows users to analyze NitroML benchmark results.<jupyter_code># This notebook assumes you have followed the following steps to setup port-forwarding:

# Step 1: Configure your cluster with gcloud
# `gcloud container clusters get-credentials <cluster_name> --zone <cluster-zone> --project <project-id>

# Step 2: Get the port where the gRPC service is running on the cluster
# `kubectl get configmap metadata-grpc-configmap -o jsonpath={.data}`
# Use `METADATA_GRPC_SERVICE_PORT` in the next step. The default port used is 8080.

# Step 3: Port forwarding
# `kubectl port-forward deployment/metadata-grpc-deployment 9898:<METADATA_GRPC_SERVICE_PORT>`

# Troubleshooting
# If getting error related to Metadata (For examples, Transaction already open). Try restarting the metadata-grpc-service using:
# `kubectl rollout restart deployment metadata-grpc-deployment`  
import sys, os
PROJECT_DIR=os.path.join(sys.path[0], '..')
%cd {PROJECT_DIR}
from ml_metadata.proto import metadata_store_pb2
from ml_metadata.metadata_store import metadata_store
from nitroml.benchmark import results<jupyter_output><empty_output><jupyter_text>Connect to the ML Metadata (MLMD) databaseFirst we need to connect to our MLMD database which stores the results of ourbenchmark runs.<jupyter_code>connection_config = metadata_store_pb2.MetadataStoreClientConfig()

connection_config.host = 'localhost'
connection_config.port = 9898

store = metadata_store.MetadataStore(connection_config)<jupyter_output><empty_output><jupyter_text>Display benchmark resultsNext we load and visualize `pd.DataFrame` containing our benchmark results.These results contain contextual features such as the pipeline ID, and benchmark metrics as computed by the downstream Evaluators. If yourbenchmark included an `EstimatorTrainer` component, its hyperparameters may alsodisplay in the table below.<jupyter_code>#@markdown ### Choose how to aggregate metrics:
mean = False  #@param { type: "boolean" }
stdev = False  #@param { type: "boolean" }
min_and_max = False  #@param { type: "boolean" }

agg = []
if mean:
    agg.append("mean")
if stdev:
    agg.append("std")
if min_and_max:
    agg += ["min", "max"]

df = results.overview(store, metric_aggregators=agg)
df.head()<jupyter_output><empty_output><jupyter_text>We can display an interactive table using qgridPlease follow the latest instructions on downloading qqgrid package from here: https://github.com/quantopian/qgrid<jupyter_code>import qgrid
qgid_wdget = qgrid.show_grid(df, show_toolbar=True)
qgid_wdget<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0847586469023185, 'fraction_numerical': 0.0068415051311288486, 'mean_word_length': 6.3314763231197775, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '10848', 'license': 'Apache-2.0', 'n_tokens_mistral': 820, 'n_tokens_neox': 788, 'n_words': 253, 'path': 'examples/kubeflow_metadata_example.ipynb', 'repo_name': 'google/nitroml', 'size': '2631'}