<jupyter_start><jupyter_text>Use ONNX Single Shot MultiBox Detector (SSD) In this notebook, we investigate how to use the [SSD](https://arxiv.org/abs/1512.02325) computer vision model serialized in [ONNX](https://onnx.ai/) format. For performing this experiment, we are using the "animal" dataset, which contains images of cute dogs and cats.  Setup EnvironmentBut before we start, let's set up our working environment. It will help us to keep the project directory clean. All artifacts generated within the project will be placed to the "tmp" directory and will be ignored by Git.<jupyter_code>import warnings
warnings.filterwarnings("ignore")

import cv2
import numpy as np
from PIL import Image
from utils import load_dataset, load_image, show_image

# Let's make a temporary directory for all artifacts created by this notebook.
import os
if not os.path.exists('tmp'):
    os.makedirs('tmp')<jupyter_output><empty_output><jupyter_text>Prepare DatasetFor creating the ML model we are loading the "animal" dataset which includes the following images:| Photo           | Description                  || ----------------| ---------------------------- || cat-1.jpeg      | an image of a cat            || cat-2.jpeg      | an image of a cat            || cat-3.jpeg      | an image of a cat            || cat-m.jpeg      | an image of 5 cats           || dog-1.jpeg      | an image of a dog            || dog-2.jpeg      | an image of a dog            || dog-3.jpeg      | an image of a dog            || dog-m.jpeg      | an image of 3 dogs           || dog-n-cat.jpeg  | an image of 2 cats and 1 dog |Please note, this data set is used for inference experiments only.<jupyter_code># Sets the loading dataset name.
dataset_name = 'animal'

# Loads the dataset.
dataset = load_dataset(dataset_name)<jupyter_output><empty_output><jupyter_text>Download SSD ONNX ModelUse the following [page](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/ssd) to get the URL for the SSD ONNX model.**Please note that downloading ONNX models have a large size. To prevent the repeated download of the same model on the notebook restart we check whether the model already exists. If a model exists we skip the download. To force download, go to the "tmp" directory and delete the "ssd.onnx" file.**<jupyter_code># Sets the model name.
model_name = 'ssd'

# Download the ONNX model.
if not os.path.exists(os.path.join('tmp', f'{model_name}.onnx')):
    !wget -O tmp/ssd.onnx https://github.com/onnx/models/raw/master/vision/object_detection_segmentation/ssd/model/ssd-10.onnx
else:
    print('This notebook is using an already downloaded model.')<jupyter_output>This notebook is using an already downloaded model.<jupyter_text>Define Image Pre/Past-processors  Pre-process Image<jupyter_code>def preprocess(image):
    image = image.resize((1200, 1200), Image.BILINEAR)
    image_data = np.array(image)
    image_data = np.transpose(image_data, [2, 0, 1])
    image_data = np.expand_dims(image_data, 0)
    mean_vec = np.array([0.485, 0.456, 0.406])
    stddev_vec = np.array([0.229, 0.224, 0.225])
    norm_image_data = np.zeros(image_data.shape).astype('float32')
    for i in range(image_data.shape[1]):
        norm_image_data[:,i,:,:] = (image_data[:,i,:,:]/255 - mean_vec[i]) / stddev_vec[i]
    return norm_image_data<jupyter_output><empty_output><jupyter_text>Post-process Image<jupyter_code>def postprocess(image, classes, boxes, labels, scores, score_threshold):
    # Resize boxes
    image = np.array(image)

    report = ""
    for box, label, score in zip(boxes, labels, scores):

        # Showing boxes with score > 0.7
        if score <= score_threshold:
            continue
        
        image = cv2.rectangle(
            image, 
            (int(box[0] * image.shape[1]), int(box[1] * image.shape[0])),
            (int(box[2] * image.shape[1]), int(box[3] * image.shape[0])),
            (100,0,100),
            2)
        image = cv2.putText(
            image,
            classes[label] + ':' + str(np.round(score, 2)),
            (int(box[0] * image.shape[1]), int(box[1] * image.shape[0] +10)),
            cv2.FONT_HERSHEY_TRIPLEX,
            0.5,
            (255,255,255)
        )
        report += "%s: %s: %.2f\n" % (str(box), classes[label], score)

    return image, report<jupyter_output><empty_output><jupyter_text>Segment ImagesFirst we need to load the model.<jupyter_code>import onnxruntime as rt
sess = rt.InferenceSession(os.path.join('tmp', f'{model_name}.onnx'))
classes = [line.rstrip('\n') for line in open(os.path.join('config', 'coco_classes.txt'))]<jupyter_output><empty_output><jupyter_text>Load image for segmentation.<jupyter_code>image_name = 'cat-1'
image = load_image(dataset_name, image_name)
show_image(image, model_name, 'orig', image_name, dataset_name)<jupyter_output><empty_output><jupyter_text>Preprocess image for the model input.<jupyter_code>pre_image = preprocess(image)
show_image(pre_image[0].transpose(1,2,0), model_name, 'pre', image_name, dataset_name)<jupyter_output>Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).<jupyter_text>Use ONNX model for segmentation.<jupyter_code>boxes, labels, scores = sess.run(
    None, {sess.get_inputs()[0].name: pre_image}
)<jupyter_output><empty_output><jupyter_text>Visualize the model predictions.<jupyter_code>score_threshold = 0.7
post_image, report = postprocess(
    image, classes, boxes[0], labels[0], scores[0], score_threshold)
show_image(post_image, model_name, 'post', image_name, dataset_name)
print('Selected Objects (score threshold %.2f):\n%s' % (score_threshold, report))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1255039439088519, 'fraction_numerical': 0.021384750219106047, 'mean_word_length': 3.6465798045602607, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '399368', 'license': 'MIT', 'n_tokens_mistral': 1914, 'n_tokens_neox': 1812, 'n_words': 517, 'path': 'onnx_ssd.ipynb', 'repo_name': 'mmgalushka/onnx-hello-world', 'size': '5705'}