<jupyter_start><jupyter_text>Copyright (c) Microsoft Corporation. All rights reserved.  Licensed under the MIT License.  YOLO Real-time Object Detection using ONNX on AzureMLThis example shows how to convert the TinyYOLO model from CoreML to ONNX and operationalize it as a web service using Azure Machine Learning services and the ONNX Runtime. What is ONNXONNX is an open format for representing machine learning and deep learning models. ONNX enables open and interoperable AI by enabling data scientists and developers to use the tools of their choice without worrying about lock-in and flexibility to deploy to a variety of platforms. ONNX is developed and supported by a community of partners including Microsoft, Facebook, and Amazon. For more information, explore the [ONNX website](http://onnx.ai). YOLO DetailsYou Only Look Once (YOLO) is a state-of-the-art, real-time object detection system. For more information about YOLO, please visit the [YOLO website](https://pjreddie.com/darknet/yolo/).  PrerequisitesTo make the best use of your time, make sure you have done the following:* Understand the [architecture and terms](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architecture) introduced by Azure Machine Learning* Go through the [00.configuration.ipynb](../00.configuration.ipynb) notebook to:    * install the AML SDK    * create a workspace and its configuration file (config.json)<jupyter_code># Check core SDK version number
import azureml.core

print("SDK version:", azureml.core.VERSION)<jupyter_output><empty_output><jupyter_text>Install necessary packagesYou'll need to run the following commands to use this tutorial:```shpip install coremltoolspip install onnxmltools```  Convert model to ONNXFirst we download the CoreML model. We use the CoreML model listed at https://coreml.store/tinyyolo. This may take a few minutes.<jupyter_code>!wget https://s3-us-west-2.amazonaws.com/coreml-models/TinyYOLO.mlmodel<jupyter_output><empty_output><jupyter_text>Then we use ONNXMLTools to convert the model.<jupyter_code>import onnxmltools
import coremltools

# Load a CoreML model
coreml_model = coremltools.utils.load_spec('TinyYOLO.mlmodel')

# Convert from CoreML into ONNX
onnx_model = onnxmltools.convert_coreml(coreml_model, 'TinyYOLOv2')

# Save ONNX model
onnxmltools.utils.save_model(onnx_model, 'tinyyolov2.onnx')

import os
print(os.path.getsize('tinyyolov2.onnx'))<jupyter_output><empty_output><jupyter_text>Deploying as a web service with Azure ML Load Azure ML workspaceWe begin by instantiating a workspace object from the existing workspace created earlier in the configuration notebook.<jupyter_code>from azureml.core import Workspace

ws = Workspace.from_config()
print(ws.name, ws.location, ws.resource_group, sep = '\n')<jupyter_output><empty_output><jupyter_text>Registering your model with Azure MLNow we upload the model and register it in the workspace.<jupyter_code>from azureml.core.model import Model

model = Model.register(model_path = "tinyyolov2.onnx",
                       model_name = "tinyyolov2",
                       tags = {"onnx": "demo"},
                       description = "TinyYOLO",
                       workspace = ws)<jupyter_output><empty_output><jupyter_text>Displaying your registered modelsYou can optionally list out all the models that you have registered in this workspace.<jupyter_code>models = ws.models()
for m in models:
    print("Name:", m.name,"\tVersion:", m.version, "\tDescription:", m.description, m.tags)<jupyter_output><empty_output><jupyter_text>Write scoring fileWe are now going to deploy our ONNX model on Azure ML using the ONNX Runtime. We begin by writing a score.py file that will be invoked by the web service call. The `init()` function is called once when the container is started so we load the model using the ONNX Runtime into a global session object.<jupyter_code>%%writefile score.py
import json
import time
import sys
import os
from azureml.core.model import Model
import numpy as np    # we're going to use numpy to process input and output data
import onnxruntime    # to inference ONNX models, we use the ONNX Runtime

def init():
    global session
    model = Model.get_model_path(model_name = 'tinyyolov2')
    session = onnxruntime.InferenceSession(model)

def preprocess(input_data_json):
    # convert the JSON data into the tensor input
    return np.array(json.loads(input_data_json)['data']).astype('float32')

def postprocess(result):
    return np.array(result).tolist()

def run(input_data_json):
    try:
        start = time.time()   # start timer
        input_data = preprocess(input_data_json)
        input_name = session.get_inputs()[0].name  # get the id of the first input of the model   
        result = session.run([], {input_name: input_data})
        end = time.time()     # stop timer
        return {"result": postprocess(result),
                "time": end - start}
    except Exception as e:
        result = str(e)
        return {"error": result}<jupyter_output><empty_output><jupyter_text>Create container imageFirst we create a YAML file that specifies which dependencies we would like to see in our container.<jupyter_code>from azureml.core.conda_dependencies import CondaDependencies 

myenv = CondaDependencies.create(pip_packages=["numpy","onnxruntime"])

with open("myenv.yml","w") as f:
    f.write(myenv.serialize_to_string())<jupyter_output><empty_output><jupyter_text>Then we have Azure ML create the container. This step will likely take a few minutes.<jupyter_code>from azureml.core.image import ContainerImage

image_config = ContainerImage.image_configuration(execution_script = "score.py",
                                                  runtime = "python",
                                                  conda_file = "myenv.yml",
                                                  description = "TinyYOLO ONNX Demo",
                                                  tags = {"demo": "onnx"}
                                                 )


image = ContainerImage.create(name = "onnxyolo",
                              models = [model],
                              image_config = image_config,
                              workspace = ws)

image.wait_for_creation(show_output = True)<jupyter_output><empty_output><jupyter_text>In case you need to debug your code, the next line of code accesses the log file.<jupyter_code>print(image.image_build_log_uri)<jupyter_output><empty_output><jupyter_text>We're all set! Let's get our model chugging. Deploy the container image<jupyter_code>from azureml.core.webservice import AciWebservice

aciconfig = AciWebservice.deploy_configuration(cpu_cores = 1, 
                                               memory_gb = 1, 
                                               tags = {'demo': 'onnx'}, 
                                               description = 'web service for TinyYOLO ONNX model')<jupyter_output><empty_output><jupyter_text>The following cell will likely take a few minutes to run as well.<jupyter_code>from azureml.core.webservice import Webservice
from random import randint

aci_service_name = 'onnx-tinyyolo'+str(randint(0,100))
print("Service", aci_service_name)

aci_service = Webservice.deploy_from_image(deployment_config = aciconfig,
                                           image = image,
                                           name = aci_service_name,
                                           workspace = ws)

aci_service.wait_for_deployment(True)
print(aci_service.state)<jupyter_output><empty_output><jupyter_text>In case the deployment fails, you can check the logs. Make sure to delete your aci_service before trying again.<jupyter_code>if aci_service.state != 'Healthy':
    # run this command for debugging.
    print(aci_service.get_logs())
    aci_service.delete()<jupyter_output><empty_output><jupyter_text>Success!If you've made it this far, you've deployed a working web service that does object detection using an ONNX model. You can get the URL for the webservice with the code below.<jupyter_code>print(aci_service.scoring_uri)<jupyter_output><empty_output><jupyter_text>When you are eventually done using the web service, remember to delete it.<jupyter_code>#aci_service.delete()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07882069795427196, 'fraction_numerical': 0.002527075812274368, 'mean_word_length': 3.4972943722943723, 'pattern_counts': {'":': 5, '<': 61, '<?xml version=': 0, '>': 61, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 24, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '94153', 'license': 'MIT', 'n_tokens_mistral': 2362, 'n_tokens_neox': 2297, 'n_words': 747, 'path': 'onnx/onnx-convert-aml-deploy-tinyyolo.ipynb', 'repo_name': 'hhy37/MachineLearningNotebooks', 'size': '8310'}