<jupyter_start><jupyter_text>Basic scoring pipeline  This notebooks is just an addition to `create_basic_training_pipeline.ipynb` to demonstrate how to use the trained model to make scoring<jupyter_code>from azureml.core import Workspace, Dataset
from azureml.core.datastore import Datastore
from azureml.core.compute import AmlCompute
from azureml.core.compute import ComputeTarget
from azureml.pipeline.steps import PythonScriptStep
from azureml.pipeline.core import PipelineData
from azureml.pipeline.core import Pipeline
from azureml.core import Experiment
from msrest.exceptions import HttpOperationError<jupyter_output><empty_output><jupyter_text>Because we are going to list all files and send them to the Form Recognizer one by one, we will need couple more parameters compare to training pipeline:- storage_name: a storage name that contains input data- storage_key: a storage key to get access to the storage with input data- container_name: the name of the container that contains folder with input data<jupyter_code>subscription_id = "<provide it here>"
wrksp_name = "<provide it here>"
resource_group = "<provide it here>"
compute_name = "mycluster"
min_nodes = 0
max_nodes = 4
vm_priority = "lowpriority"
vm_size = "Standard_F2s_v2"
project_folder = "basic_scoring_steps"
fr_endpoint = "<provide it here>"
fr_key = "<provide it here>"
storage_name = "<provide it here>"
storage_key = "<provide it here>"
container_name = "data"
datastore_name = "data_ds"
scoring_ds_name = "basic_scoring"<jupyter_output><empty_output><jupyter_text>Getting a reference to the workspace. If it doesn't exist there is no sense to create new one because we don't have any models anyway<jupyter_code>try:
    aml_workspace = Workspace.get(
        name=wrksp_name,
        subscription_id=subscription_id,
        resource_group=resource_group)
    print("Found the existing Workspace")
except Exception as e:
    print(f"Workspace doesn't exist")<jupyter_output><empty_output><jupyter_text>Create a compute cluster for scoring<jupyter_code>if compute_name in aml_workspace.compute_targets:
    compute_target = aml_workspace.compute_targets[compute_name]
    if compute_target and type(compute_target) is AmlCompute:
        print(f"Found existing compute target {compute_name} so using it")
else:
    compute_config = AmlCompute.provisioning_configuration(
        vm_size=vm_size,
        vm_priority=vm_priority,
        min_nodes=min_nodes,
        max_nodes=max_nodes,
    )

    compute_target = ComputeTarget.create(aml_workspace, compute_name,
                                                  compute_config)
    compute_target.wait_for_completion(show_output=True)<jupyter_output><empty_output><jupyter_text>We need to mount blob storage with input data tp scoring compute cluster. To do that we need to register blob container as a data store in AML<jupyter_code>try:
    blob_datastore = Datastore.get(aml_workspace, datastore_name)
    print("Found Blob Datastore with name: %s" % datastore_name)
except HttpOperationError:
    blob_datastore = Datastore.register_azure_blob_container(
        workspace=aml_workspace,
        datastore_name=datastore_name,
        account_name=storage_name,
        container_name=container_name,
        account_key=storage_key)
print("Registered blob datastore with name: %s" % datastore_name)<jupyter_output><empty_output><jupyter_text>Now, we can create a dataset with all input files there (it doesn't make much sense for this example, but it's very useful for parallel step)<jupyter_code>scoring_file_path = blob_datastore.path("Test")
scoring_file_dataset = Dataset.File.from_files(path=scoring_file_path, validate=True)
scoring_file_dataset = scoring_file_dataset.register(
    aml_workspace, scoring_ds_name, create_new_version=True)
print("Dataset has been registered")<jupyter_output><empty_output><jupyter_text>We need pipeline data object to store all outputs from scoring<jupyter_code>scoring_output = PipelineData(
    "scoring_output",
    datastore=blob_datastore)<jupyter_output><empty_output><jupyter_text>Just one step here: scoring. We will list all files and make scoring one by one<jupyter_code>scoring_step = PythonScriptStep(
    name = "scoring",
    script_name="score.py",
    inputs=[scoring_file_dataset.as_named_input("scoring_files")],
    outputs=[scoring_output],
    arguments=[
        "--output", scoring_output,
        "--fr_endpoint", fr_endpoint,
        "--fr_key", fr_key],
    compute_target=compute_target,
    source_directory=project_folder
)
steps = [scoring_step]<jupyter_output><empty_output><jupyter_text>Create a pipeline object with one step only<jupyter_code>pipeline = Pipeline(workspace=aml_workspace, steps=steps)<jupyter_output><empty_output><jupyter_text>Execute the pipeline<jupyter_code>pipeline_run = Experiment(aml_workspace, 'scoring_basic_exp').submit(pipeline)
pipeline_run.wait_for_completion()<jupyter_output><empty_output><jupyter_text>Register the pipeline as an reusable entity in AML<jupyter_code>pipeline.publish(
    name="basic_scoring",
    description="Scoring data using form recognizer single model")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07590759075907591, 'fraction_numerical': 0.0007765482430596001, 'mean_word_length': 5.7700394218134035, 'pattern_counts': {'":': 0, '<': 52, '<?xml version=': 0, '>': 52, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '513885', 'license': 'MIT', 'n_tokens_mistral': 1586, 'n_tokens_neox': 1570, 'n_words': 405, 'path': 'Pipelines/Azure_Machine_Learning/create_basic_scoring_pipeline.ipynb', 'repo_name': 'shanepeckham/knowledge-extraction-recipes-forms-1', 'size': '5151'}