<jupyter_start><jupyter_text>Create a CPU Accelerated YOLOv3 Inference EngineIn this section, we will create an inference engine wrapper, a class that will get image data as input, analyze it, and return the analysis result. If you do not wish to create a custom inference engine wrapper, you can use a pre-built YOLOv3 HTTP model from Microsoft Container Registry by altering the deployment.common.template.json file. Under the lvaExtension node, change the ```"image": "$CONTAINER_REGISTRY_USERNAME_myacr.azurecr.io/$CONTAINER_IMAGE_NAME"```  to ```"image": "mcr.microsoft.com/lva-utilities/lvaextension:http-yolov3-onnx-v1.0"```Then, skip this step (10. Create an inference engine) and the next two steps (11. Containerize inference engine solution and 12. Upload container image to ACR), and go directly to step 13. If you want to run other samples in the future, first revert the deployment.common.template.json file to its original content.  Get Global VariablesFirst, read the previously stored variables. We need the name of the directory that will be used to store our ML solution files. If this directory does not exist, we will create a directory with a specified directory name.<jupyter_code>import sys
sys.path.append('../../../common')
from env_variables import *

import os
if not os.path.exists(lvaExtensionPath):
    os.mkdir(lvaExtensionPath)<jupyter_output><empty_output><jupyter_text>Download the ONNX ML ModelNext, we will download the sample ONNX model to use in the solution. Recall that in this sample, we will be using YOLOv3, an object detection ML model. However, you may customize this ML model to fit your needs.<jupyter_code># Do not change the values below, as the names are embedded into the score.py file.
# If the values are changed, you must also update the score.py content according to new file names
onnxModelFileName = "model.onnx"
onnxLabelFileName = "labels.txt"

onnxModelUrl = "https://media.githubusercontent.com/media/onnx/models/master/vision/object_detection_segmentation/yolov3/model/yolov3-10.onnx"
onnxModelLabels = "https://raw.githubusercontent.com/qqwweee/keras-yolo3/master/model_data/coco_classes.txt"<jupyter_output><empty_output><jupyter_text>The code snippet below downloads the YOLOv3 model from the [ONNX Model Zoo](https://github.com/onnx/models) repository. > [!NOTE]> The model download URLs frequently change, so if any of the code below fails, please update the source URLs accordingly.<jupyter_code># Download the model files
import urllib.request
import os

# Download the Yolo V3 pre-trained model
lvaExtensionFilePath = os.path.join(lvaExtensionPath, onnxModelFileName)
if not os.path.exists(lvaExtensionFilePath):
    res = urllib.request.urlretrieve(onnxModelUrl, lvaExtensionFilePath)
    print("Model file downloaded at: {}".format(lvaExtensionFilePath))
else:
    print("{} already exists here, so not downloading again.".format(lvaExtensionFilePath))
    
# Download the labels of the Yolo V3 pre-trained model
lvaExtensionLabelFilePath = os.path.join(lvaExtensionPath, onnxLabelFileName)
if not os.path.exists(lvaExtensionLabelFilePath):
    res = urllib.request.urlretrieve(onnxModelLabels, lvaExtensionLabelFilePath)
    print("Labels file downloaded at: {}".format(lvaExtensionLabelFilePath))
else:
    print("{} already exists here, so not downloading again.".format(lvaExtensionLabelFilePath))<jupyter_output><empty_output><jupyter_text>Create Inference Engine WrapperNext, we will create a class that with different properties and methods to help scoring and analysing data from an image. This class will also help us specify analytics compute targets, such as CPU, VPU, FPGA, and debugging features.  >  [!IMPORTANT] > Specific to this sample, we are using the YOLOv3 model. As you can see from the code below,  > * The YOLOv3 model accepts only raw image bytes with 416 by 416 in size.  > * Because we expect the `score` method to receive raw bytes of this size (416x416), we have statically coded the image size into our code. If the image is not 416x416 float32, then the code will crash.> * Why do we statically code the image size? LVA sends video frames to the `score` endpoint. In fact, LVA can send any image size and format. Since LVA can send images with 416x416 size, we do not need to spend additional compute cycles for re-sizing an image.<jupyter_code>%%writefile $lvaExtensionPath/score.py
import threading
import cv2
import numpy as np
import io
import onnxruntime
import json
import logging
import os
import linecache
import sys

logging.basicConfig(level=logging.DEBUG)

def PrintGetExceptionDetails():
    exType, exValue, exTraceback = sys.exc_info()

    tbFrame = exTraceback.tb_frame
    lineNo = exTraceback.tb_lineno
    fileName = tbFrame.f_code.co_filename

    linecache.checkcache(fileName)
    line = linecache.getline(fileName, lineNo, tbFrame.f_globals)

    exMessage = '[LVAX] Exception:\n\tFile name: {0}\n\tLine number: {1}\n\tLine: {2}\n\tValue: {3}'.format(fileName, lineNo, line.strip(), exValue)

    logging.info(exMessage)


class MLModel:
    def __init__(self):
        try:
            self._modelFileName = 'model.onnx'
            self._labelFileName = 'labels.txt'
            self._lock = threading.Lock()

            with open(self._labelFileName, "r") as f:
                self._labelList = [l.rstrip() for l in f]
            
            self._onnxSession = onnxruntime.InferenceSession(self._modelFileName)

        except:
            PrintGetExceptionDetails()

    def Preprocess(self, cvImage):
        try:
            imageBlob = cv2.cvtColor(cvImage, cv2.COLOR_BGR2RGB)
            imageBlob = np.array(imageBlob, dtype='float32')
            imageBlob /= 255.
            imageBlob = np.transpose(imageBlob, [2, 0, 1])
            imageBlob = np.expand_dims(imageBlob, 0)

            return imageBlob
        except:
            PrintGetExceptionDetails()

    def Postprocess(self, boxes, scores, indices):
        try:
            detectedObjects = []

            for idx in indices:
                idxTuple = (idx[0], idx[2])
                temp = [i for i in boxes[idxTuple]]  # temp[1, 0, 3, 2] = xmin, ymin, xmax, ymax
                dobj = {
                    "type" : "entity",
                    "entity" : {
                        "tag" : {
                            "value" : self._labelList[idx[1]],
                            "confidence" : str(scores[tuple(idx)])
                        },
                        "box" : {
                            "l" : str(temp[1] / 416),
                            "t" : str(temp[0] / 416),
                            "w" : str((temp[3] - temp[1]) / 416),
                            "h" : str((temp[2] - temp[0]) / 416)
                        }
                    }
                }
                detectedObjects.append(dobj)
            return detectedObjects
            
        except:
            PrintGetExceptionDetails()

    def Score(self, cvImage):
        try:
            with self._lock:
                imageBlob = self.Preprocess(cvImage)
                boxes, scores, indices = self._onnxSession.run(None, {"input_1": imageBlob, "image_shape":np.array([[416, 416]], dtype=np.float32)})
        
            return self.Postprocess(boxes, scores, indices)

        except:
            PrintGetExceptionDetails()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08888888888888889, 'fraction_numerical': 0.013685636856368564, 'mean_word_length': 3.013594344752583, 'pattern_counts': {'":': 4, '<': 17, '<?xml version=': 0, '>': 24, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '207429', 'license': 'MIT', 'n_tokens_mistral': 2172, 'n_tokens_neox': 2067, 'n_words': 700, 'path': 'utilities/video-analysis/notebooks/Yolo/yolov3/yolov3-http-icpu-onnx/create_yolov3_icpu_inference_engine.ipynb', 'repo_name': 'fvneerden/live-video-analytics', 'size': '7380'}