<jupyter_start><jupyter_text>Local Inference with trained ALBERT model Let's first import all the required packages.<jupyter_code>from metaflow import Flow, get_metadata
from determined.experimental import Determined
import torch<jupyter_output><empty_output><jupyter_text>We can interact with Metaflow directly and return the most recent run of the DeterminedFlow:<jupyter_code>run = Flow('DeterminedFlow').latest_successful_run
print(f"Using run: {str(run)}")<jupyter_output><empty_output><jupyter_text>Populate Determined fields with Metaflow run Since our DeterminedFlow tracks the experiment run on Determined, we can easily retrieve the experiment id and the cluster url that the experiment was run on.<jupyter_code>master_url = run.data.det_master
experiment_id = run.data.experiment_id

print(f"master_url: {master_url}")
print(f"experiment_id: {experiment_id}")<jupyter_output><empty_output><jupyter_text>Load best performing checkpoint from Determined Using this information, we can retrieve the top checkpoint for the experiment...<jupyter_code>checkpoint = Determined(master=master_url). \
    get_experiment(experiment_id). \
    top_checkpoint()<jupyter_output><empty_output><jupyter_text>...and load it as a model for running inference.<jupyter_code>model = checkpoint.load(map_location=torch.device('cpu'))<jupyter_output><empty_output><jupyter_text>Use model for prediction locally We'll first import our predict function.<jupyter_code>from predict import run_prediction<jupyter_output><empty_output><jupyter_text>Then set the context and question...<jupyter_code>context = "New Zealand is a sovereign island country in the southwestern Pacific Ocean. It has a total land area of 268,000 square kilometres and a population of 4.9 million."
questions = ["How many people live in New Zealand?"]<jupyter_output><empty_output><jupyter_text>...and finally run the prediction:<jupyter_code>predictions = run_prediction(model, questions, context)
for key in predictions.keys():
    print(predictions[key])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08459657701711491, 'fraction_numerical': 0.003911980440097799, 'mean_word_length': 7.934497816593886, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '565027', 'license': 'Apache-2.0', 'n_tokens_mistral': 620, 'n_tokens_neox': 601, 'n_words': 174, 'path': 'metaflow/local-inference.ipynb', 'repo_name': 'hoanghphan/works-with-determined', 'size': '2045'}