<jupyter_start><jupyter_text>Object detectionThe objective is to locate objects in the image. Our model will predict the coordinates of a box that encloses the object.<jupyter_code>import mxnet as mx
import gluoncv as gcv
import matplotlib.pyplot as plt
mx.__version__
gcv.__version__<jupyter_output><empty_output><jupyter_text>Download de imageThis will save the image in the local path of Jupyter installation:<jupyter_code>image_url = 'https://raw.githubusercontent.com/zhreshold/mxnet-ssd/master/data/demo/dog.jpg'
image_file_path = 'dog.jpg'
gcv.utils.download(url=image_url, path=image_file_path)<jupyter_output>Downloading dog.jpg from https://raw.githubusercontent.com/zhreshold/mxnet-ssd/master/data/demo/dog.jpg...<jupyter_text>Load the image<jupyter_code>image = mx.image.imread('dog.jpg')
print(f'image type: {type(image)}')
print(f'image shape: {image.shape}')
print(f'image data type: {image.dtype}')
print(f'image, minimum value: {image.min().asscalar()}')
print(f'image, maximum value: {image.max().asscalar()}')<jupyter_output>image type: <class 'mxnet.ndarray.ndarray.NDArray'>
image shape: (576, 768, 3)
image data type: <class 'numpy.uint8'>
image, minimum value: 0
image, maximum value: 255<jupyter_text>In the **image type** shows that image is loaded as an MXNet ndarray.In the **image shape** shows we have a data layout in HWC format, i.e. the image has a **H**eight of 576 pixels with a **W**idth of 768 pixels and it's a colored image with three **C**hannels.<jupyter_code>plt.imshow(image.asnumpy())<jupyter_output><empty_output><jupyter_text>Transform imageWe need to transform the image to send it to the network or apply the CV model on it. We need to add **N** as a fourth dimension to the HWC format to transform it to NCHW (**N**: batch **C**: channel **H**: hight **W**: width).<jupyter_code>transformed_image, chw_image = gcv.data.transforms.presets.yolo.transform_test(
    image, short=512)

print(f'image shape: {transformed_image.shape}')
print(f'image data type: {transformed_image.dtype}')
print(f'image, minimum value: {transformed_image.min().asscalar()}')
print(f'image, maximum value: {transformed_image.max().asscalar()}')<jupyter_output>image shape: (1, 3, 512, 683)
image data type: <class 'numpy.float32'>
image, minimum value: -2.1179039478302
image, maximum value: 2.640000104904175<jupyter_text>**transformed_image** variable is the image in the new format (NCHW) and the height will be resized to 512 pixels but maintaining the aspect ratio. Furthermore, it's a batch of one image. And it's an array of 32-bit floats instead of 8-bit integers.**chw_image** variable is a resized version of the image (transformed_image).  Download and load pre-trained modelWe'll use the [**YOLOv3**](https://github.com/ultralytics/yolov3) (You Only Look Once) network, which is the most popular algorithm to object detection, with a [**darknet53**](https://github.com/pjreddie/darknet) backbone that has been trained on the Microsoft's [**COCO**](https://cocodataset.org/) image dataset. Our network parameters are 237 megabytes size and are stored in the mxnet cache, i.e. ~/.mxnet/models/ is the path where we can find the downloaded models.<jupyter_code>network = gcv.model_zoo.get_model('yolo3_darknet53_coco', pretrained=True)<jupyter_output>Downloading /home/jovyan/.mxnet/models/yolo3_darknet53_coco-09767802.zip from https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/models/yolo3_darknet53_coco-09767802.zip...<jupyter_text>Make a predictionRun forward pass to obtain the predicted score for each class<jupyter_code>prediction = network(transformed_image)
print(f'Type Prediction object: {type(prediction)}')
for index, array in enumerate(prediction):
    print(f'#{index + 1} {array.shape}')<jupyter_output>#1 (1, 100, 1)
#2 (1, 100, 1)
#3 (1, 100, 4)<jupyter_text>- The first array contains the object class indexes, i.e. with a shape of (1, 100, 1), we have 1 image, 100 potential objects (the model can predict up to 100 objects in a single image), and 1 class index per object.- The second array contains the object class probabilities.- The third array contains the object bounding box coordinates, i.e. with shape (1, 100, 4), we have 1 image, 100 potential objects, and 4 values for each object to define its bounding box (which delimited the object predicted).  Remove additionalbatch dimensionRemove additional batch dimension for all of the arrays.<jupyter_code>prediction = [array[0] for array in prediction]
class_indicies, probabilities, bounding_boxes = prediction

k = 10
print(class_indicies[:k])<jupyter_output>[[16.]
 [ 1.]
 [ 7.]
 [ 2.]
 [13.]
 [ 0.]
 [-1.]
 [-1.]
 [-1.]
 [-1.]]
<NDArray 10x1 @cpu(0)><jupyter_text>The first detected object has a predicted class with number 16, and we see more objects with classes 1, 7, 2, 13 and 0. After this, we have a number of objects with a class index of -1. Where -1 is a special class index that is used to indicate there is no detected object. We have six detected objects in total, with the remaining 94 potential objects being padded with -1 values.We can show what refers to class 16, for example:<jupyter_code>class_index = 16
print(f'{network.classes[class_index]} => {class_index}')<jupyter_output>dog => 16<jupyter_text>And how many classes do we have in the network? We have 80 which are the image categories that can find in the [COCO dataset](https://cocodataset.org/detection-eval).<jupyter_code>len(network.classes)<jupyter_output><empty_output><jupyter_text>Object probabilitiesNow, we have the associated object class probability. We can interpret this as our confidence that the class index (image category) is correct.<jupyter_code>print(probabilities[:k])<jupyter_output>[[ 0.9919528 ]
 [ 0.9600399 ]
 [ 0.62269694]
 [ 0.29241997]
 [ 0.01795173]
 [ 0.01141722]
 [-1.        ]
 [-1.        ]
 [-1.        ]
 [-1.        ]]
<NDArray 10x1 @cpu(0)><jupyter_text>If we use a confidence of 50%, then we can say that the first three objects were detected. The -1 values means that there is not confidence in that classes (image categories).  Object boundingsNow, we have the coordinates of the bounding boxes for each object detected. They specify the coordinates for the top-left corner, and the bottom-right corner.<jupyter_code>print(bounding_boxes[:k])<jupyter_output>[[116.53647  201.33235  281.90323  482.09094 ]
 [ 93.92975  107.73944  504.75128  375.75415 ]
 [416.78833   69.800644 615.01794  148.89009 ]
 [416.78833   69.800644 615.01794  148.89009 ]
 [ 90.88356  125.878815 504.4449   402.7955  ]
 [532.78235   91.84185  547.3104   104.69111 ]
 [ -1.        -1.        -1.        -1.      ]
 [ -1.        -1.        -1.        -1.      ]
 [ -1.        -1.        -1.        -1.      ]
 [ -1.        -1.        -1.        -1.      ]]
<NDArray 10x4 @cpu(0)><jupyter_text>Visualize predictionNow, wen can graph the result using the resized image above.<jupyter_code>gcv.utils.viz.plot_bbox(chw_image, bounding_boxes,
                        probabilities, class_indicies, class_names=network.classes)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11422109132032796, 'fraction_numerical': 0.06389595702572802, 'mean_word_length': 4.549019607843137, 'pattern_counts': {'":': 0, '<': 50, '<?xml version=': 0, '>': 52, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '146455', 'license': 'MIT', 'n_tokens_mistral': 2610, 'n_tokens_neox': 2295, 'n_words': 762, 'path': 'demo_object_detection.ipynb', 'repo_name': 'edycop/aws_cv_python', 'size': '7074'}