<jupyter_start><jupyter_text>Deploy Object Detection Model Use ModelCIMMDetction is a well-known open source object detection toolbox based on PyTorch. You can refer to  for more details.By walking through this tutorial, you will be able to:- Load pretained MMDetction model- Convert MMDetction model into ONNX format - Register and retrieve models by ModelHub 1. Prequisities  1.1 Installation of MMDetction  Firstly you have to install MMDetction according to official instructions :<jupyter_code>!pip install https://download.openmmlab.com/mmcv/dist/latest/torch1.7.0/cu102/mmcv_full-latest%2Btorch1.7.0%2Bcu102-cp37-cp37m-manylinux1_x86_64.whl
!git clone https://github.com/open-mmlab/mmdetection.git
!cd mmdetection && pip install -q -r requirements/build.txt && pip install -q -v -e .<jupyter_output>Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Obtaining file:///home/sherry/project/github/ML-Model-CI/example/notebook/mmdetection
Requirement already satisfied: matplotlib in /home/sherry/project/miniconda3/envs/test/lib/python3.7/site-packages (from mmdet==2.7.0) (3.3.3)
Requirement already satisfied: mmpycocotools in /home/sherry/project/miniconda3/envs/test/lib/python3.7/site-packages (from mmdet==2.7.0) (12.0.3)
Requirement already satisfied: numpy in /home/sherry/project/miniconda3/envs/test/lib/python3.7/site-packages (from mmdet==2.7.0) (1.19.2)
Requirement already satisfied: six in /home/sherry/project/miniconda3/envs/test/lib/python3.7/site-packages (from mmdet==2.7.0) (1.15.0)
Requirement already satisfied: terminaltables in /home/sherry/project/miniconda3/envs/test/lib/python3.7/site-packages (from mmdet==2.7.0) (3.1.0)
Requirement already satisfied: python-dateutil>=2.1 in /home/sherry/project/miniconda3/envs/test/lib/python3.7/site-packages (from matplotli[...]<jupyter_text>1.2 Start ModelCI ServiceThen we can start our ModelCI service, you should at least set mongodb password before starting. You can refer to [last notebook](https://github.com/cap-ntu/ML-Model-CI/blob/master/example/notebook/image_classification_model_deployment.ipynb) for more details.<jupyter_code>%env MONGO_PASSWORD=<PASSWORD>
!conda activate modelci && modelci start<jupyter_output>env: MONGO_PASSWORD=<PASSWORD>
2020-12-24 09:56:20,119 - ml-modelci Docker Container Manager - INFO - Container name=mongo-99022 stared
2020-12-24 09:56:21,831 - ml-modelci Docker Container Manager - INFO - Container name=cadvisor-75216 started.
2020-12-24 09:56:26,296 - ml-modelci Docker Container Manager - INFO - Container name=dcgm-exporter-29627 started.
2020-12-24 09:56:28,282 - ml-modelci Docker Container Manager - INFO - gpu-metrics-exporter-28682 stared
2020-12-24 09:56:28,571 - modelci backend - INFO - Uvicorn server listening on 8000<jupyter_text>2. Build MMdetection Model 2.1 ImportsWe should import the following functions:- preprocess_example_input: for generating tensor and meta info from example image file- build_model_from_cfg: for building model form config file and checkpoint file<jupyter_code>from mmdet.core import preprocess_example_input, build_model_from_cfg<jupyter_output><empty_output><jupyter_text>2.2 Model ConfigWe should either use a dict or config file for configuration of MMDetection model, to make things simple, we use a config file provided by MMDetection.Notice: - You may need to manually download pretrained model checkpoints from [MMDetection models zoo](https://github.com/open-mmlab/mmdetection/blob/master/docs/model_zoo.md).- Only a few MMdet models are able to converted into ONNX format, you can refer to [documentation](https://mmdetection.readthedocs.io/en/latest/tutorials/pytorch2onnx.htmllist-of-supported-models-exportable-to-onnx) for more detail.<jupyter_code>config_file = 'mmdetection/configs/retinanet/retinanet_r50_fpn_1x_coco.py'
checkpoint_file = 'retinanet_r50_fpn_1x_coco_20200130-c2398f9e.pth'<jupyter_output><empty_output><jupyter_text>2.3 Build ModelThen we can build our MMdetection model based on the configuration above and the checkpoint file we already download.<jupyter_code>model = build_model_from_cfg(config_file, checkpoint_file)<jupyter_output><empty_output><jupyter_text>Before conversion, we need to modify forward function to provide the necessary **kwargs parameters such as img_metas.In order to obtain valid bbox data during the onnx tracing process, we also need to use a tensor generated from image file as model input instead of random tensors.<jupyter_code>input_config = {
    'input_shape': (1,3,224,224),
    'input_path': 'mmdetection/demo/demo.jpg',
    'normalize_cfg': {
        'mean': (123.675, 116.28, 103.53),
        'std': (58.395, 57.12, 57.375)
        }
}
one_img, one_meta = preprocess_example_input(input_config)
one_img.shape
from functools import partial
model.forward = partial(model.forward, img_metas=[[one_meta]], return_loss=False)<jupyter_output><empty_output><jupyter_text>3. Register ModelWe can convert the pytorch model above into optimized formats, such as ONNX through modelci 3.1 Imports- modelci.hub.manager: for registering model into ModelHub- modelci.types.bo: for constructing model inputs paramenters- modelci.types.trtis_objects: for specifying model input shape format<jupyter_code>from modelci.hub.manager import register_model
from modelci.types.bo import IOShape, Framework, Task, Engine, Metric, ModelVersion
from modelci.types.trtis_objects import ModelInputFormat<jupyter_output><empty_output><jupyter_text>3.2 Sepcify Input and OutputHere are some parameters need to be specified before model conversion.- inputs: The model inputs info- outputs: The model outputs info<jupyter_code>inputs = [IOShape([-1, 3, 204, 204], dtype=float, name='IMAGE', format=ModelInputFormat.FORMAT_NCHW)]
outputs = [
    IOShape([-1, 100, 5], dtype=float, name='BBOX'),
    IOShape([-1, 100], dtype=float, name='SCORE')
    ]<jupyter_output><empty_output><jupyter_text>3.3 RegisterIn this step, the inputs and outputs variable generated before are reused.<jupyter_code>register_model(
    model,
    dataset='COCO',
    task=Task.OBJECT_DETECTION,
    metric={Metric.MAP: 0.365},
    outputs=outputs,
    inputs=inputs,
    architecture='RetinaNet',
    framework=Framework.PYTORCH,
    version=ModelVersion('1'),
    model_input=[one_img],
    profile=False
)<jupyter_output>2021-01-13 21:54:15,831 - converter - WARNING - This model is not supported as torchscript format
2021-01-13 21:54:23,234 - converter - INFO - ONNX format converted successfully<jupyter_text>As we could see, MLModelCI support auto conversion of PyTorch models into both torchscript and ONNX format, as a result.However, this model cannot be transformed into torchscript format, but supportive of ONNX format conversion, there could be serveral factors contributing to model conversion failture such as the model structure and code format.  4. Retrieve ModelThe following steps will retrieve the model we just registered.<jupyter_code>from modelci.hub.manager import retrieve_model
retrieved_models = retrieve_model(
        architecture_name = 'RetinaNet',
        framework = Framework.PYTORCH,
        version=ModelVersion('1')
)
retrieved_models<jupyter_output><empty_output><jupyter_text>It's no wonder we get two model objects here cause there is an addition ONNX format model created automatically during previous registering process.<jupyter_code>retrieved_models[0].__dict__
retrieved_models[1].__dict__<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09408133136491746, 'fraction_numerical': 0.043215675748221716, 'mean_word_length': 5.588859416445623, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 45, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '489184', 'license': 'Apache-2.0', 'n_tokens_mistral': 2561, 'n_tokens_neox': 2351, 'n_words': 652, 'path': 'example/notebook/object_detection_model_deployment.ipynb', 'repo_name': 'PengZhang0/ML-Model-CI', 'size': '7455'}