<jupyter_start><jupyter_text>**Set Environment Variables**<jupyter_code># Use Cell Magic to Set Environment Variables
%env PROJECT=your-project-id
%env YOUR_GCS_BUCKET=your-bucket-name<jupyter_output><empty_output><jupyter_text>**Install All Dependencies**This tutorial requires the use of Tensorflow Object Detection. https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md<jupyter_code># Install Python Dependencies for Tensorflow Object Detection
!pip3 install --user Cython
!pip3 install --user contextlib2
!pip3 install --user pillow
!pip3 install --user lxml
!pip3 install --user jupyter
!pip3 install --user matplotlib
!pip3 install --user tensorflow --upgrade
# Download Required Source from Github
%cd /content/datalab
!git clone https://github.com/cocodataset/cocoapi.git
!git clone https://github.com/tensorflow/tensorflow.git
!git clone https://github.com/tensorflow/models.git
# COCO API Installation
%cd /content/datalab/cocoapi/PythonAPI
!make
!cp -r pycocotools /content/datalab/models/research/
# Manual protobuf-compiler installation and usage
%cd /content/datalab/models/research
!wget -O protobuf.zip https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip
!unzip protobuf.zip
!./bin/protoc object_detection/protos/*.proto --python_out=.
# Add Libraries to PYTHONPATH
%cd /content/datalab/models/research
%env PYTHONPATH=$PYTHONPATH:/content/datalab/models/research:/content/datalab/models/research/slim<jupyter_output><empty_output><jupyter_text>**Testing Tensorflow Object Detection**You should see output similar to this after running the next command.![alt text](https://screenshot.googleplex.com/NQ2CpJVEoGJ.png)<jupyter_code># Testing the Installation of Tensorflow Object Detection
!python3 -m object_detection.builders.model_builder_test<jupyter_output><empty_output><jupyter_text>**Setting up the Dataset**To keep things simple, we’ll use a small pet breeds dataset. This dataset includes around 7,400 images — ~200 images for 37 different cat and dog breeds. Each image has an associated annotations file, which includes the bounding box coordinates where the specific pet is located in the image. We can’t feed these images and annotations directly to our model; we need to convert them into a format our model can understand. For this we’ll use the TFRecord format.The Oxford-IIIT Pet data set is located: http://www.robots.ox.ac.uk/~vgg/data/pets/. To download, extract and convert it to TFRecrods, run the following commands below.<jupyter_code># Download and Extract Images
%cd /content/datalab/models/research
!wget http://www.robots.ox.ac.uk/~vgg/data/pets/data/images.tar.gz
!wget http://www.robots.ox.ac.uk/~vgg/data/pets/data/annotations.tar.gz
!tar -xvf annotations.tar.gz
!tar -xvf images.tar.gz
# Create TFRecords from Downloaded Images. Note that some examples may be ignored.
%cd /content/datalab/models/research
!mkdir /content/datalab/preprocess_output
!python3 -m object_detection.dataset_tools.create_pet_tf_record\
    --label_map_path=/content/datalab/models/research/object_detection/data/pet_label_map.pbtxt \
    --data_dir=/content/datalab/models/research \
    --output_dir=/content/datalab/preprocess_output
# Copy Data Files to Google Cloud Storage 
!gsutil -m cp -r /content/datalab/preprocess_output/* gs://$YOUR_GCS_BUCKET/data/
!gsutil cp /content/datalab/models/research/object_detection/data/pet_label_map.pbtxt gs://$YOUR_GCS_BUCKET/data/pet_label_map.pbtxt<jupyter_output><empty_output><jupyter_text>**Using the SSD MobileNet Checkpoint for Transfer Learning**Training a model to recognize pet breeds from scratch would take thousands of training images for each pet breed and hours or days of training time. To speed this up, we can make use of transfer learning — a process where we take the weights of a model that has already been trained on lots of data to perform a similar task, and then train the model on our own data, fine tuning the layers from the pre-trained model.There are many models we can use that have been trained to recognize a wide variety of objects in images. We can use the checkpoints from these trained models and then apply them to our custom object detection task. This works because, to a machine, the task of identifying the pixels in an image that contain basic objects like tables, chairs, or cats isn’t so different from identifying the pixels in an image that contain specific pet breeds.For this example we’ll use SSD with MobileNet, an object detection model optimized for inference on mobile. First, download and extract the latest MobileNet checkpoint that’s been pretrained on the COCO dataset. To see a list of all the models that the Object Detection API supports, check out the model zoo: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md. Once you’ve extracted the checkpoint, copy the 3 files into your GCS bucket. Run the commands below to download the checkpoint and copy it into your bucket.<jupyter_code># Copy Model Checkpoints
%cd /tmp
!curl -O http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_0.75_depth_300x300_coco14_sync_2018_07_03.tar.gz
!tar xzf ssd_mobilenet_v1_0.75_depth_300x300_coco14_sync_2018_07_03.tar.gz
!gsutil cp /tmp/ssd_mobilenet_v1_0.75_depth_300x300_coco14_sync_2018_07_03/model.ckpt.* gs://$YOUR_GCS_BUCKET/data/<jupyter_output><empty_output><jupyter_text>**Training a Quantized Model with Cloud TPU on Cloud Machine Learning Engine**Machine learning models have two distinct computational components: training and inference. In this example, we’re making use of Cloud TPUs to accelerate training. There are a few lines in the config file that relate specifically to TPU training. We can use a larger batch size when training on TPUs since they make it easier to handle large datasets (when experimenting with batch size on your own dataset, make sure to use multiples of 8 since data needs to be divided evenly for each of the 8 TPU cores). With a larger batch size for our model, we can reduce the number of training steps (in this example we use 2000). The focal loss function we use for this training job, defined in the following lines in the config, is also a great fit for TPUs.![alt text](https://screenshot.googleplex.com/LkdPseifqst.png)This loss function computes loss for every example in the dataset and then reweights them, assigning more relative weight to hard, misclassified examples. This logic is better suited for TPUs than the hard example mining operation used in other training jobs. You can read more about focal loss here: https://arxiv.org/abs/1708.02002.Recall from above that the process of initializing a pre-trained model checkpoint and then adding our own training data is called transfer learning. The following lines in the config tell our model that we’ll be doing transfer learning for object detection, starting from a pre-trained checkpoint.![alt text](https://screenshot.googleplex.com/OuJ9oNqmaxr.png)We also need to consider how our model will be used after it’s been trained. Let’s say our pet detector becomes a global hit, used by animal lovers and pet stores everywhere. We need a scalable way to handle these inference requests with low latency. The output of a machine learning model is a binary file containing the trained weights of our model — these files are often quite large, but since we’ll be serving this model directly on a mobile device we’ll need to make it as small as possible.This is where model quantization comes in. Quantization compresses the weights and activations in our model to an 8-bit fixed point representation. The following lines in our config file will generate a quantized model:![alt text](https://screenshot.googleplex.com/RQVtBcvFWxo.png)We will download this premade config but take time to examine the contents which will be print after running the commands below.To tell ML Engine where to find our training and test files and model checkpoint, you’ll need to update a few lines in the config file we’ve created for you to point to your bucket. The code below will do this for us.We will download this premade config but take time to examine the contents which will be print after running the commands below. The config file will be uploaded to Google Cloud Storage in the command below for use with Cloud Machine Learning Engine.<jupyter_code># The sed command is used to replace YOUR-STORAGE-BUCKET 
# in the config file with the name of your GCS Bucket
#!git clone .config
!sed -i 's@YOUR-STORAGE-BUCKET@'"$YOUR_GCS_BUCKET"'@g' /content/datalab/cloud-cmle-mobilenet/ssd_mobilenet_v1_0.75_depth_quantized_300x300_pets_sync.config
!cat /content/datalab/cloud-cmle-mobilenet/ssd_mobilenet_v1_0.75_depth_quantized_300x300_pets_sync.config
!gsutil cp /content/datalab/cloud-cmle-mobilenet/ssd_mobilenet_v1_0.75_depth_quantized_300x300_pets_sync.config gs://$YOUR_GCS_BUCKET/data/pipeline.config<jupyter_output><empty_output><jupyter_text>Before we kick off our training job on Cloud ML Engine, we need to package the Object Detection API, pycocotools, and TF Slim. We can do that with the following command (run this from the research/ directory, and note that the parentheses are part of the command):<jupyter_code># Package Files for Training on Cloud Machine Learning Engine
%cd /content/datalab/models/research
!bash object_detection/dataset_tools/create_pycocotools_package.sh /tmp/pycocotools
!python setup.py sdist
!(cd slim && python setup.py sdist)<jupyter_output><empty_output><jupyter_text>**Start a Training Job on Cloud Machine Learning Engine**We’re ready to train our model! To kick off training, run the following gcloud command.Note that if you receive an error saying that no Cloud TPUs are available, we recommend simply trying again in a different zone (Cloud TPUs are currently available in us-central1-b, us-central1-c, europe-west4-a, and asia-east1-c).<jupyter_code># Submit Training Job
%cd /content/datalab/models/research
!gcloud ml-engine jobs submit training `whoami`_object_detection_`date +%s` \
--job-dir=gs://$YOUR_GCS_BUCKET/train \
--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \
--module-name object_detection.model_tpu_main \
--python-version 3.5 \
--runtime-version 1.9 \
--scale-tier BASIC_TPU \
--region us-central1 \
-- \
--model_dir=gs://$YOUR_GCS_BUCKET/train \
--tpu_zone us-central1 \
--pipeline_config_path=gs://$YOUR_GCS_BUCKET/data/pipeline.config<jupyter_output><empty_output><jupyter_text>You can view the submitted job in the Google Cloud Console by navigating to **ML Engine > Jobs**. We will continue to the next section without waiting for the job to finish. It will take about 30 minutes.![alt text](https://screenshot.googleplex.com/PNL97SifydR.png) **Install the Bazel Build Tool**<jupyter_code># Download Bazel Installer
%cd /content/datalab
!wget https://github.com/bazelbuild/bazel/releases/download/0.19.1/bazel-0.19.1-installer-linux-x86_64.sh
# Run the installer
%cd /content/datalab
!chmod +x bazel-0.19.1-installer-linux-x86_64.sh
!./bazel-0.19.1-installer-linux-x86_64.sh --user<jupyter_output><empty_output><jupyter_text>**Converting the Model to TFLite** Since we are not waiting on the model to finish training, let's the model checkpoints from a model that was previously trained. These checkpoints were included in a git repo we downloaded earlier.<jupyter_code># Set Environment Variables
CONFIG_FILE="gs://$YOUR_GCS_BUCKET/data/pipeline.config"
CHECKPOINT_PATH="/content/datalab/cloud-cmle-mobilenet/model_checkpoints/model.ckpt-2000"
OUTPUT_DIR="/content/datalab/frozen_graph"
!echo {CONFIG_FILE}
# Freeze Tensorflow Graph
%env PYTHONPATH=$PYTHONPATH:/content/datalab/models/research:/content/datalab/models/research/slim
%cd /content/datalab/models/research
!python3 -m object_detection.export_tflite_ssd_graph \
--pipeline_config_path={CONFIG_FILE} \
--trained_checkpoint_prefix={CHECKPOINT_PATH} \
--output_directory={OUTPUT_DIR} \
--add_postprocessing_op=true<jupyter_output><empty_output><jupyter_text>The following command will create the tflite file that can be used for mobile or edge deployment. It will take about 10 minutes to run and is not necessary for completing the lab.<jupyter_code># Create TFLite Model File for Mobile Deployment
# This command will take about 10 minutes to complete on an 8 core machine.
%cd /content/datalab/tensorflow
!/content/bin/bazel run -c opt tensorflow/lite/toco:toco -- \
--input_file={OUTPUT_DIR}/tflite_graph.pb \
--output_file=/content/datalab/detect.tflite \
--input_shapes=1,300,300,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3'  \
--inference_type=QUANTIZED_UINT8 \
--mean_values=128 \
--std_values=128 \
--change_concat_input_ranges=false \
--allow_custom_ops<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0806736906867646, 'fraction_numerical': 0.016611551180496807, 'mean_word_length': 5.3279805352798055, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 46, 'https://': 13, 'lorem ipsum': 0, 'www.': 3, 'xml': 1}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '361151', 'license': 'Apache-2.0', 'n_tokens_mistral': 4147, 'n_tokens_neox': 3895, 'n_words': 1389, 'path': 'Train_MobileNet_on_Cloud_TPU.ipynb', 'repo_name': 'gmikels/cloud-cmle-mobilenet', 'size': '13003'}