<jupyter_start><jupyter_text>TEMPLATE TO FILL/EVOLVE Deploying a Real-Time Inferencing Service in AML compute With '<NAME>' model published in Workspace model registry.  Model originally created with the Pipeline-AutoMLStep-based approach.So far we've been training, operationalizing the training and registering a model. Now it's time to deploy a model as a real-time service that client applications can use to get predictions from new data.  Connect to Your WorkspaceThe first thing you need to do is to connect to your workspace using the Azure ML SDK.> **Note**: If the authenticated session with your Azure subscription has expired since you completed the previous exercise, you'll be prompted to reauthenticate.<jupyter_code>import azureml.core
from azureml.core import Workspace

# Load the workspace from the saved config file
ws = Workspace.from_config()
print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))<jupyter_output><empty_output><jupyter_text>Deploy a Registered Model as a Web Service (In AML Inference compute) In previous challenge steps you have trained and registered a ML model that classifies whether a client will make a claim in the upcoming year or not. This model can now be used in a production environment such as an insurance company. To support this scenario, you will deploy the model as a web service.First, let's determine what models you have registered in the workspace.<jupyter_code>from azureml.core import Model

for model in Model.list(ws):
    print(model.name, 'version:', model.version)
    for tag_name in model.tags:
        tag = model.tags[tag_name]
        print ('\t',tag_name, ':', tag)
    for prop_name in model.properties:
        prop = model.properties[prop_name]
        print ('\t',prop_name, ':', prop)
    print('\n')<jupyter_output><empty_output><jupyter_text>Right, now let's get the model that we want to deploy. By default, if we specify a model name, the latest version will be returned.<jupyter_code>model = ws.models['porto-model-from-automlstep']
print(model.name, 'version', model.version)<jupyter_output><empty_output><jupyter_text>We're going to create a web service to host this model, and this will require some code and configuration files; so let's create a folder for those.<jupyter_code>import os

folder_name = 'porto-service'

# Create a folder for the web service files
experiment_folder = './' + folder_name
os.makedirs(folder_name, exist_ok=True)

print(folder_name, 'folder created.')<jupyter_output><empty_output><jupyter_text>The service *entry script* The web service where we deploy the model will need some Python code to load the input data, get the model from the workspace, and generate and return predictions. We'll save this code in an *entry script* that will be deployed to the web service:<jupyter_code>%%writefile $folder_name/score.py
import json
import joblib
import numpy as np
import pandas as pd
from azureml.core.model import Model

# Called when the service is loaded
def init():
    global model
    # Get the path to the deployed model file and load it
    model_path = Model.get_model_path("porto-model-from-automlstep")
    model = joblib.load(model_path)

# Called when a request is received
def run(raw_data):
    # Get the input data as a numpy array  
    
    numpy_data = np.array(json.loads(raw_data)['data'])

    # HARD-CODED row
    # numpy_data = np.array([[20,2,1,3,1,0,0,1,0,0,0,0,0,0,0,8,1,0,0,0.6,0.1,0.61745445,6,1,-1,0,1,11,1,1,0,1,99,2,0.31622777,0.6396829,0.36878178,3.16227766,0.2,0.6,0.5,2,2,8,1,8,3,10,3,0,0,10,0,1,0,0,1,0]])

    # Convert to Pandas DataFrame and add the column names to the DataFrame
    df_data = pd.DataFrame(data=numpy_data, columns=['id', 'ps_ind_01', 'ps_ind_02_cat', 'ps_ind_03', 'ps_ind_04_cat',
                                                   'ps_ind_05_cat', 'ps_ind_06_bin', 'ps_ind_07_bin', 'ps_ind_08_bin',
                                                   'ps_ind_09_bin', 'ps_ind_10_bin', 'ps_ind_11_bin', 'ps_ind_12_bin',
                                                   'ps_ind_13_bin', 'ps_ind_14', 'ps_ind_15', 'ps_ind_16_bin',
                                                   'ps_ind_17_bin', 'ps_ind_18_bin', 'ps_reg_01', 'ps_reg_02', 'ps_reg_03',
                                                   'ps_car_01_cat', 'ps_car_02_cat', 'ps_car_03_cat', 'ps_car_04_cat',
                                                   'ps_car_05_cat', 'ps_car_06_cat', 'ps_car_07_cat', 'ps_car_08_cat',
                                                   'ps_car_09_cat', 'ps_car_10_cat', 'ps_car_11_cat', 'ps_car_11',
                                                   'ps_car_12', 'ps_car_13', 'ps_car_14', 'ps_car_15', 'ps_calc_01',
                                                   'ps_calc_02', 'ps_calc_03', 'ps_calc_04', 'ps_calc_05', 'ps_calc_06',
                                                   'ps_calc_07', 'ps_calc_08', 'ps_calc_09', 'ps_calc_10', 'ps_calc_11',
                                                   'ps_calc_12', 'ps_calc_13', 'ps_calc_14', 'ps_calc_15_bin',
                                                   'ps_calc_16_bin', 'ps_calc_17_bin', 'ps_calc_18_bin', 'ps_calc_19_bin',
                                                   'ps_calc_20_bin'])
       
    # Get a prediction from the model
    predictions = model.predict(df_data)
    
    # Get the corresponding classname for each prediction (0 or 1)
    classnames = ['no-claim', 'claim']
    predicted_classes = []
    for prediction in predictions:
        predicted_classes.append(classnames[prediction])
    
    # Return the predictions as JSON
    return json.dumps(predicted_classes)<jupyter_output><empty_output><jupyter_text>Use the AutoML curated environmentThe web service will be hosted in a container, and the container will need to install any required Python dependencies when it gets initialized. In this case, our scoring code requires a valid Conda environment for AutoML generated models that tells the container host to install this into the environment.<jupyter_code>from azureml.core import Environment

envs = Environment.list(workspace=ws)

# List Environments and packages in my workspace
for env in envs:
    if env.startswith("AzureML"):
        print("Name",env)
        #print("packages", envs[env].python.conda_dependencies.serialize_to_string())
        
# Use curated environment from AML named "AzureML-Tutorial"
automl_curated_environment = Environment.get(workspace=ws, name="AzureML-AutoML")<jupyter_output><empty_output><jupyter_text>Now you're ready to deploy and we'll deploy the container you created above. The deployment process includes the following steps:1. Define an inference configuration, which includes the scoring and environment files required to load and use the model.2. Define a deployment configuration that defines the execution environment in which the service will be hosted. In this case, an Azure Container Instance.3. Deploy the model as a web service.4. Verify the status of the deployed service.> **More Information**: For more details about model deployment, and options for target execution environments, see the [documentation](https://docs.microsoft.com/azure/machine-learning/how-to-deploy-and-where).Deployment will take some time as it first runs a process to create a container image, and then runs a process to create a web service based on the image. When deployment has completed successfully, you'll see a status of **Healthy**.<jupyter_code>from azureml.core.webservice import AciWebservice
from azureml.core.model import InferenceConfig

# CODE TBD - Code the InferenceConfig object and AciWebservice.deploy_configuration() to deploy the model in ACI
#
#
#
#<jupyter_output><empty_output><jupyter_text>Hopefully, the deployment has been successful and you can see a status of **Healthy**. If not, you can use the following code to check the status and get the service logs to help you troubleshoot.<jupyter_code># If you need to make a change and redeploy, you may need to delete unhealthy service using the following code:

# service.delete()

print(service.state)
# print(service.get_logs())<jupyter_output><empty_output><jupyter_text>For more information about publishing a model as a service, see the [documentation](https://docs.microsoft.com/azure/machine-learning/how-to-deploy-and-where)  Try the Model Service already deployed in AMLAfter your model is deployed, perform a call to the web service using service.run().Test the deployed service by submitting a REST request to its endpoint abd review the predictions it returns.Use the following test data in numpy array format. This represents details for two drivers, for which your service should predict the likelihood of an insurance claim.[[0,1,8,1,0,0,1,0,0,0,0,0,0,0,12,1,0,0,0.5,0.3,0.610327781,7,1,-1,0,-1,1,1,1,2,1,65,1,0.316227766,0.669556409,0.352136337,3.464101615,0.1,0.8,0.6,1,1,6,3,6,2,9,1,1,1,12,0,1,1,0,0,1],    [4,2,5,1,0,0,0,0,1,0,0,0,0,0,5,1,0,0,0.9,0.5,0.771362431,4,1,-1,0,0,11,1,1,0,1,103,1,0.316227766,0.60632002,0.358329457,2.828427125,0.4,0.5,0.4,3,3,8,4,10,2,7,2,0,3,10,0,0,1,1,0,1]]<jupyter_code># Prepare data for predictions to try (3 rows --> 3 Predictions)

import json

input_payload = json.dumps({
    'data': [[20,2,1,3,1,0,0,1,0,0,0,0,0,0,0,8,1,0,0,0.6,0.1,0.61745445,6,1,-1,0,1,11,1,1,0,1,99,2,0.31622777,0.6396829,0.36878178,3.16227766,0.2,0.6,0.5,2,2,8,1,8,3,10,3,0,0,10,0,1,0,0,1,0],
             [0,1,8,1,0,0,1,0,0,0,0,0,0,0,12,1,0,0,0.5,0.3,0.610327781,7,1,-1,0,-1,1,1,1,2,1,65,1,0.316227766,0.669556409,0.352136337,3.464101615,0.1,0.8,0.6,1,1,6,3,6,2,9,1,1,1,12,0,1,1,0,0,1,0],
             [4,2,5,1,0,0,0,0,1,0,0,0,0,0,5,1,0,0,0.9,0.5,0.771362431,4,1,-1,0,0,11,1,1,0,1,103,1,0.316227766,0.60632002,0.358329457,2.828427125,0.4,0.5,0.4,3,3,8,4,10,2,7,2,0,3,10,0,0,1,1,0,1,0]],
    'method': 'predict'  # If you have a classification model, you can get probabilities by changing this to 'predict_proba'.
})
# Get service from Workspace
from azureml.core import Webservice

service = Webservice(workspace=ws, name='porto-services')

print(service.scoring_uri)
print(service.swagger_uri)<jupyter_output><empty_output><jupyter_text>Do predictions with the service<jupyter_code>output = service.run(input_payload)

print(output) # It should return several predictions, as many as rows in the provided hard-coded data
service.get_logs()<jupyter_output><empty_output><jupyter_text>When you are finished testing your service, clean up the deployment with service.delete().<jupyter_code># service.delete()<jupyter_output><empty_output><jupyter_text>Package service as Docker imageIn some cases, you might want to create a Docker image without deploying the model (if, for example, you plan to deploy to Azure App Service, Kubernetes cluster or a simple Docker Host such as a development PC with Docker installed. You might even want to download the files used to build the image, inspect them, modify them, and build the image manually.Model packaging enables you to do these things. It packages all the assets needed to host a model as a web service and allows you to download either a fully built Docker image or the files needed to build one. There are two ways to use model packaging:- Download a packaged model: Download a Docker image that contains the model and other files needed to host it as a web service.- Generate a Dockerfile: Download the Dockerfile, model, entry script, and other assets needed to build a Docker image. You can then inspect the files or make changes before you build the image locally.References:https://docs.microsoft.com/en-us/azure/machine-learning/how-to-deploy-and-wherepackage-models The following is a simple packaging code. The output image will be pushed to your workspace's ACR.You must include an Environment object in your inference configuration to use Model.package().<jupyter_code>from azureml.core.model import Model
package = Model.package(ws, [model], inference_config)
package.wait_for_creation(show_output=True)  # Or show_output=False to hide the Docker build logs.
package.pull()<jupyter_output><empty_output><jupyter_text>Instead of a fully-built image, you can also generate a Dockerfile and download all the assets needed to build a Docker image in your custom Docker environment.<jupyter_code>package = Model.package(ws, [model], inference_config, generate_dockerfile=True)
package.wait_for_creation(show_output=True)
package.save("./local_context_dir")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1027364378300528, 'fraction_numerical': 0.06240998559769563, 'mean_word_length': 4.13305954825462, 'pattern_counts': {'":': 0, '<': 54, '<?xml version=': 0, '>': 57, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 32, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '422361', 'license': 'MIT', 'n_tokens_mistral': 4263, 'n_tokens_neox': 3955, 'n_words': 1222, 'path': 'notebooks/Challenge03.B/Challenge_3.B_deploy_service_and_inference_TEMPLATE.ipynb', 'repo_name': 'shoresh57/AzureMLWorkshop', 'size': '12504'}