<jupyter_start><jupyter_text>Copyright (c) Microsoft Corporation. All rights reserved.  Licensed under the MIT License. ![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/MachineLearningNotebooks/how-to-use-azureml/deployment/onnx/onnx-modelzoo-aml-deploy-resnet50.png)  ResNet50 Image Classification using ONNX and AzureMLThis example shows how to deploy the ResNet50 ONNX model as a web service using Azure Machine Learning services and the ONNX Runtime. What is ONNXONNX is an open format for representing machine learning and deep learning models. ONNX enables open and interoperable AI by enabling data scientists and developers to use the tools of their choice without worrying about lock-in and flexibility to deploy to a variety of platforms. ONNX is developed and supported by a community of partners including Microsoft, Facebook, and Amazon. For more information, explore the [ONNX website](http://onnx.ai). ResNet50 DetailsResNet classifies the major object in an input image into a set of 1000 pre-defined classes. For more information about the ResNet50 model and how it was created can be found on the [ONNX Model Zoo github](https://github.com/onnx/models/tree/master/vision/classification/resnet).   PrerequisitesTo make the best use of your time, make sure you have done the following:* Understand the [architecture and terms](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architecture) introduced by Azure Machine Learning* If you are using an Azure Machine Learning Notebook VM, you are all set.  Otherwise, go through the [configuration notebook](../../../configuration.ipynb) to:    * install the AML SDK    * create a workspace and its configuration file (config.json)<jupyter_code># Check core SDK version number
import azureml.core

print("SDK version:", azureml.core.VERSION)<jupyter_output><empty_output><jupyter_text>Download pre-trained ONNX model from ONNX Model Zoo.Download the [ResNet50v2 model and test data](https://s3.amazonaws.com/onnx-model-zoo/resnet/resnet50v2/resnet50v2.tar.gz) and extract it in the same folder as this tutorial notebook.<jupyter_code>import urllib.request

onnx_model_url = "https://s3.amazonaws.com/onnx-model-zoo/resnet/resnet50v2/resnet50v2.tar.gz"
urllib.request.urlretrieve(onnx_model_url, filename="resnet50v2.tar.gz")

!tar xvzf resnet50v2.tar.gz<jupyter_output><empty_output><jupyter_text>Deploying as a web service with Azure ML  Load your Azure ML workspaceWe begin by instantiating a workspace object from the existing workspace created earlier in the configuration notebook.<jupyter_code>from azureml.core import Workspace

ws = Workspace.from_config()
print(ws.name, ws.location, ws.resource_group, sep = '\n')<jupyter_output><empty_output><jupyter_text>Register your model with Azure MLNow we upload the model and register it in the workspace.<jupyter_code>from azureml.core.model import Model

model = Model.register(model_path = "resnet50v2/resnet50v2.onnx",
                       model_name = "resnet50v2",
                       tags = {"onnx": "demo"},
                       description = "ResNet50v2 from ONNX Model Zoo",
                       workspace = ws)<jupyter_output><empty_output><jupyter_text>Displaying your registered modelsYou can optionally list out all the models that you have registered in this workspace.<jupyter_code>models = ws.models
for name, m in models.items():
    print("Name:", name,"\tVersion:", m.version, "\tDescription:", m.description, m.tags)<jupyter_output><empty_output><jupyter_text>Write scoring fileWe are now going to deploy our ONNX model on Azure ML using the ONNX Runtime. We begin by writing a score.py file that will be invoked by the web service call. The `init()` function is called once when the container is started so we load the model using the ONNX Runtime into a global session object.<jupyter_code>%%writefile score.py
import json
import time
import sys
import os
import numpy as np    # we're going to use numpy to process input and output data
import onnxruntime    # to inference ONNX models, we use the ONNX Runtime

def softmax(x):
    x = x.reshape(-1)
    e_x = np.exp(x - np.max(x))
    return e_x / e_x.sum(axis=0)

def init():
    global session
    # AZUREML_MODEL_DIR is an environment variable created during deployment.
    # It is the path to the model folder (./azureml-models/$MODEL_NAME/$VERSION)
    # For multiple models, it points to the folder containing all deployed models (./azureml-models)
    model = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'resnet50v2.onnx')
    session = onnxruntime.InferenceSession(model, None)

def preprocess(input_data_json):
    # convert the JSON data into the tensor input
    img_data = np.array(json.loads(input_data_json)['data']).astype('float32')
    
    #normalize
    mean_vec = np.array([0.485, 0.456, 0.406])
    stddev_vec = np.array([0.229, 0.224, 0.225])
    norm_img_data = np.zeros(img_data.shape).astype('float32')
    for i in range(img_data.shape[0]):
        norm_img_data[i,:,:] = (img_data[i,:,:]/255 - mean_vec[i]) / stddev_vec[i]

    return norm_img_data

def postprocess(result):
    return softmax(np.array(result)).tolist()

def run(input_data_json):
    try:
        start = time.time()
        # load in our data which is expected as NCHW 224x224 image
        input_data = preprocess(input_data_json)
        input_name = session.get_inputs()[0].name  # get the id of the first input of the model   
        result = session.run([], {input_name: input_data})
        end = time.time()     # stop timer
        return {"result": postprocess(result),
                "time": end - start}
    except Exception as e:
        result = str(e)
        return {"error": result}<jupyter_output><empty_output><jupyter_text>Create inference configuration First we create a YAML file that specifies which dependencies we would like to see in our container.<jupyter_code>from azureml.core.conda_dependencies import CondaDependencies 


myenv = CondaDependencies.create(pip_packages=["numpy", "onnxruntime", "azureml-core", "azureml-defaults"])

with open("myenv.yml","w") as f:
    f.write(myenv.serialize_to_string())<jupyter_output><empty_output><jupyter_text>Create the inference configuration object. Please note that you must indicate azureml-defaults with verion >= 1.0.45 as a pip dependency, because it contains the functionality needed to host the model as a web service.<jupyter_code>from azureml.core.model import InferenceConfig
from azureml.core.environment import Environment


myenv = Environment.from_conda_specification(name="myenv", file_path="myenv.yml")
inference_config = InferenceConfig(entry_script="score.py", environment=myenv)<jupyter_output><empty_output><jupyter_text>Deploy the model<jupyter_code>from azureml.core.webservice import AciWebservice

aciconfig = AciWebservice.deploy_configuration(cpu_cores = 1, 
                                               memory_gb = 1, 
                                               tags = {'demo': 'onnx'}, 
                                               description = 'web service for ResNet50 ONNX model')<jupyter_output><empty_output><jupyter_text>The following cell will likely take a few minutes to run as well.<jupyter_code>from random import randint

aci_service_name = 'onnx-demo-resnet50'+str(randint(0,100))
print("Service", aci_service_name)
aci_service = Model.deploy(ws, aci_service_name, [model], inference_config, aciconfig)
aci_service.wait_for_deployment(True)
print(aci_service.state)<jupyter_output><empty_output><jupyter_text>In case the deployment fails, you can check the logs. Make sure to delete your aci_service before trying again.<jupyter_code>if aci_service.state != 'Healthy':
    # run this command for debugging.
    print(aci_service.get_logs())
    aci_service.delete()<jupyter_output><empty_output><jupyter_text>Success!If you've made it this far, you've deployed a working web service that does image classification using an ONNX model. You can get the URL for the webservice with the code below.<jupyter_code>print(aci_service.scoring_uri)<jupyter_output><empty_output><jupyter_text>When you are eventually done using the web service, remember to delete it.<jupyter_code>aci_service.delete()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08890760346487006, 'fraction_numerical': 0.014557266602502407, 'mean_word_length': 4.419165580182529, 'pattern_counts': {'":': 4, '<': 53, '<?xml version=': 0, '>': 54, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '47952', 'license': 'MIT', 'n_tokens_mistral': 2572, 'n_tokens_neox': 2440, 'n_words': 778, 'path': 'how-to-use-azureml/deployment/onnx/onnx-modelzoo-aml-deploy-resnet50.ipynb', 'repo_name': 'aydensutt/MachineLearningNotebooks', 'size': '8312'}