<jupyter_start><jupyter_text>Copyright 2021 The TensorFlow Authors.<jupyter_code>#@title Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.<jupyter_output><empty_output><jupyter_text>Tutorial: Convert models trained using TensorFlow Object Detection API to TensorFlow LiteThis tutorial demonstrate these steps:* Convert TensorFlow models trained using the TensorFlow Object Detection API to [TensorFlow Lite](https://www.tensorflow.org/lite).* Add the required metadata using [TFLite Metadata Writer API](https://www.tensorflow.org/lite/convert/metadata_writer_tutorialobject_detectors). This will make the TFLite model compatible with [TFLite Task Library](https://www.tensorflow.org/lite/inference_with_metadata/task_library/object_detector), so that the model can be integrated in mobile apps in 3 lines of code.       Run in Google Colab        View source on GitHub  Preparation  Install the TFLite Support Library<jupyter_code>!pip install -q tflite_support<jupyter_output><empty_output><jupyter_text>Install the TensorFlow Object Detection API<jupyter_code>import os
import pathlib

# Clone the tensorflow models repository if it doesn't already exist
if "models" in pathlib.Path.cwd().parts:
  while "models" in pathlib.Path.cwd().parts:
    os.chdir('..')
elif not pathlib.Path('models').exists():
  !git clone --depth 1 https://github.com/tensorflow/models
%%bash
cd models/research/
protoc object_detection/protos/*.proto --python_out=.
cp object_detection/packages/tf2/setup.py .
pip install -q .<jupyter_output><empty_output><jupyter_text>Import the necessary libraries<jupyter_code>import matplotlib
import matplotlib.pyplot as plt

import os
import random
import io
import imageio
import glob
import scipy.misc
import numpy as np
from six import BytesIO
from PIL import Image, ImageDraw, ImageFont
from IPython.display import display, Javascript
from IPython.display import Image as IPyImage

import tensorflow as tf

from object_detection.utils import label_map_util
from object_detection.utils import config_util
from object_detection.utils import visualization_utils as viz_utils
from object_detection.utils import colab_utils
from object_detection.utils import config_util
from object_detection.builders import model_builder

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Download a pretrained model from Model ZooIn this tutorial, we demonstrate converting a pretrained model `SSD MobileNet V2 FPNLite 640x640` in the [TensorFlow 2 Model Zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md). You can replace the model with your own model and the rest will work the same.<jupyter_code>!wget http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8.tar.gz
!tar -xf ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8.tar.gz
!rm ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8.tar.gz<jupyter_output><empty_output><jupyter_text>Generate TensorFlow Lite Model  Step 1: Export TFLite inference graphFirst, we invoke `export_tflite_graph_tf2.py` to generate a TFLite-friendly intermediate SavedModel. This will then be passed to the TensorFlow Lite Converter for generating the final model.Use `--help` with the above script to get the full list of supported parameters.These can fine-tune accuracy and speed for your model.<jupyter_code>!python models/research/object_detection/export_tflite_graph_tf2.py \
    --trained_checkpoint_dir {'ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/checkpoint'} \
    --output_directory {'ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/tflite'} \
    --pipeline_config_path {'ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/pipeline.config'}<jupyter_output><empty_output><jupyter_text>Step 2: Convert to TFLiteUse the [TensorFlow Lite Converter](https://www.tensorflow.org/lite/convert) toconvert the `SavedModel` to TFLite. Note that you need to use `from_saved_model`for TFLite conversion with the Python API.You can also leverage[Post-training Quantization](https://www.tensorflow.org/lite/performance/post_training_quantization)to[optimize performance](https://www.tensorflow.org/lite/performance/model_optimization)and obtain a smaller model. In this tutorial, we use the [dynamic range quantization](https://www.tensorflow.org/lite/performance/post_training_quant).<jupyter_code>_TFLITE_MODEL_PATH = "ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/model.tflite"

converter = tf.lite.TFLiteConverter.from_saved_model('ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/tflite/saved_model')
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()

with open(_TFLITE_MODEL_PATH, 'wb') as f:
  f.write(tflite_model)<jupyter_output><empty_output><jupyter_text>Step 3: Add MetadataThe model needs to be packed with [TFLite Metadata](https://www.tensorflow.org/lite/convert/metadata) to enable easy integration into mobile apps using the [TFLite Task Library](https://www.tensorflow.org/lite/inference_with_metadata/task_library/object_detector). This metadata helps the inference code perform the correct pre & post processing as required by the model. Use the following code to create the metadata.<jupyter_code># Download the COCO dataset label map that was used to trained the SSD MobileNet V2 FPNLite 640x640 model
!wget https://raw.githubusercontent.com/tensorflow/models/master/research/object_detection/data/mscoco_label_map.pbtxt -q

# We need to convert the Object Detection API's labelmap into what the Task API needs:
# a txt file with one class name on each line from index 0 to N.
# The first '0' class indicates the background.
# This code assumes COCO detection which has 90 classes, you can write a label
# map file for your model if re-trained.
_ODT_LABEL_MAP_PATH = 'mscoco_label_map.pbtxt'
_TFLITE_LABEL_PATH = "ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/tflite_label_map.txt"

category_index = label_map_util.create_category_index_from_labelmap(
    _ODT_LABEL_MAP_PATH)
f = open(_TFLITE_LABEL_PATH, 'w')
for class_id in range(1, 91):
  if class_id not in category_index:
    f.write('???\n')
    continue
  name = category_index[class_id]['name']
  f.write(name+'\n')
f.close()<jupyter_output><empty_output><jupyter_text>Then we'll add the label map and other necessary metadata (e.g. normalization config) to the TFLite model.As the `SSD MobileNet V2 FPNLite 640x640` model take input image with pixel value in the range of [-1..1] ([code](https://github.com/tensorflow/models/blob/b09e75828e2c65ead9e624a5c7afed8d214247aa/research/object_detection/models/ssd_mobilenet_v2_keras_feature_extractor.pyL132)), we need to set `norm_mean = 127.5` and `norm_std = 127.5`. See this [documentation](https://www.tensorflow.org/lite/convert/metadatanormalization_and_quantization_parameters) for more details on the normalization parameters.<jupyter_code>from tflite_support.metadata_writers import object_detector
from tflite_support.metadata_writers import writer_utils

_TFLITE_MODEL_WITH_METADATA_PATH = "ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8/model_with_metadata.tflite"

writer = object_detector.MetadataWriter.create_for_inference(
    writer_utils.load_file(_TFLITE_MODEL_PATH), input_norm_mean=[127.5], 
    input_norm_std=[127.5], label_file_paths=[_TFLITE_LABEL_PATH])
writer_utils.save_file(writer.populate(), _TFLITE_MODEL_WITH_METADATA_PATH)<jupyter_output><empty_output><jupyter_text>Optional: Print out the metadata added to the TFLite model.<jupyter_code>from tflite_support import metadata

displayer = metadata.MetadataDisplayer.with_model_file(_TFLITE_MODEL_WITH_METADATA_PATH)
print("Metadata populated:")
print(displayer.get_metadata_json())
print("=============================")
print("Associated file(s) populated:")
print(displayer.get_packed_associated_file_list())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08325324363286882, 'fraction_numerical': 0.024147044690052858, 'mean_word_length': 6.637614678899083, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 15, 'lorem ipsum': 0, 'www.': 11, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '121104', 'license': 'Apache-2.0', 'n_tokens_mistral': 2827, 'n_tokens_neox': 2637, 'n_words': 698, 'path': 'research/object_detection/colab_tutorials/convert_odt_model_to_TFLite.ipynb', 'repo_name': 'wwkiyyx/models', 'size': '8324'}