<jupyter_start><jupyter_text>[Colaboratory]Tensorflow2-ObjectDetectionAPI-Colab-Hands-On.ipynbこのノートブックはTensorflow2 Object Detection APIの学習/推論のハンズオン用スクリプトです(This notebook is a hands-on script for learning/inference of the Tensorflow2 Object Detection API)Colaboratoryのハードウェア アクセラレータ設定をGPUにして実行してください(Set the hardware accelerator setting of Colaboratory to GPU and execute it.)Github URL : [Tensorflow2-ObjectDetectionAPI-Colab-Hands-On](https://github.com/Kazuhito00/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On)  Google Driveマウント(Google Drive mount)※checkpoint、saved model格納先 (checkpoint, saved model storage location)<jupyter_code>from google.colab import drive
drive.mount('./gdrive')<jupyter_output><empty_output><jupyter_text>Tensorflow Object Detection API設定(Set Tensorflow Object Detection API)  Protocol Buffers<jupyter_code>!curl -OL https://github.com/google/protobuf/releases/download/v3.2.0/protoc-3.2.0-linux-x86_64.zip
!unzip protoc-3.2.0-linux-x86_64.zip -d protoc3
!sudo mv protoc3/bin/* /usr/local/bin/
!sudo mv protoc3/include/* /usr/local/include/
!rm -rf protoc3 protoc-3.2.0-linux-x86_64.zip
!git clone --depth 1 https://github.com/tensorflow/models
%cd /content/models/research

!/usr/local/bin/protoc object_detection/protos/*.proto --python_out=.<jupyter_output><empty_output><jupyter_text>必要ライブラリインストール(Installation of required libraries)<jupyter_code>!cp /content/models/research/object_detection/packages/tf2/setup.py .
!python -m pip install .
# インストール成否確認(Confirmation of successful installation)
!python /content/models/research/object_detection/builders/model_builder_tf2_test.py<jupyter_output><empty_output><jupyter_text>Tensorflow2-ObjectDetectionAPI-Colab-Hands-Onリポジトリクローン(Clone repository)<jupyter_code>!git clone https://github.com/Kazuhito00/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On<jupyter_output><empty_output><jupyter_text>TFRecordをアップロード(Upload TF Record)「Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/02_tfrecord」にVoTTからエクスポートしたTFRecordとtf_label_map.pbtxtを格納してください。(Please store the TFRecord and tf_label_map.pbtxt exported from VoTT  in "Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/02_tfrecord".)![](https://user-images.githubusercontent.com/37477845/94039064-31f8cc80-fe02-11ea-81f2-28427d759099.png)  学習データ/検証データ 分割(Split Training data/validation data)<jupyter_code>original_data_dir = '/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/02_tfrecord'
train_data_dir = '/content/models/research/train_data'
val_data_dir = '/content/models/research/val_data'
# ディレクトリ作成(Create a directory)
import os
import shutil

shutil.rmtree(train_data_dir, ignore_errors=True)
shutil.rmtree(val_data_dir, ignore_errors=True)
os.mkdir(train_data_dir)
os.mkdir(val_data_dir)
import glob

# ファイル数カウント(Count the number of files)
file_count = len(glob.glob(original_data_dir + '/*.tfrecord'))
print('File count : ' + str(file_count))
import random

# 学習データ/検証データ 分割(Split Training data/validation data.)
train_ratio = 0.75

file_list = glob.glob(original_data_dir + '/*.tfrecord')
random_sample_list = random.sample(file_list, file_count)

# ディレクトリへコピー(Copy to directory)
for index, filepath in enumerate(random_sample_list):
    if index < int(file_count * train_ratio):
        # 学習データ(Training data)
        shutil.copy2(filepath, train_data_dir)
    else:
        # 検証データ(Validation data)
        shutil.copy2(filepath, val_data_dir)<jupyter_output><empty_output><jupyter_text>学習済モデル(Pre-Trained model)このハンズオンでは「Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/03_pretrained_model」に学習済モデル(EfficientDet-D0)が格納してあります。(In this hands-on, the pre-trained model(EfficientDet-D0) is stored in "Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/03_pretrained_model")学習済モデル取得元(Pre-Trained model acquisition source)：http://download.tensorflow.org/models/object_detection/tf2/20200711/efficientdet_d0_coco17_tpu-32.tar.gz  パイプラインコンフィグアップロード(Pipeline config upload)パイプラインコンフィグを修正し「Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/03_pretrained_model」にアップロードしてください(Please modify the pipeline config and upload it to "Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/03_pretrained_model")パイプラインコンフィグは以下の行を修正します(Pipeline Config modifies the following line)* 3行目(Line 3)：クラス数(num_classes)変更前(Before) : 90変更後(After) : 1* 134行目(Line 134)：バッチサイズ(batch_size)変更前(Before) : 128変更後(After) : 16* 161行目(Line 161)：ファインチューニング用のチェックポイント格納先(fine_tune_checkpoint)変更前(Before) : "PATH_TO_BE_CONFIGURED"変更後(After) : "/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/03_pretrained_model/efficientdet_d0_coco17_tpu-32/checkpoint/ckpt-0"* 167行目(Line 167)：ファインチューニング方法(fine_tune_checkpoint_type)変更前(Before) : "classification"変更後(After) : "detection"* 168行目(Line 168)：Googleカスタム 16ビットbrain浮動小数点の使用有無(use_bfloat16)変更前(Before) : true変更後(After) : false* 172行目(Line 172)：ラベルマップファイルの格納先(label_map_path)変更前(Before) : "PATH_TO_BE_CONFIGURED/label_map.txt"変更後(After) : "/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/02_tfrecord/tf_label_map.pbtxt"* 174行目(Line 174)：学習データの格納先(input_path)変更前(Before) : "PATH_TO_BE_CONFIGURED/train2017-?????-of-00256.tfrecord"変更後(After) : "/content/models/research/train_data/??????.tfrecord"* 185行目(Line 185)：ラベルマップファイルの格納先(label_map_path)変更前(Before) : "PATH_TO_BE_CONFIGURED/label_map.txt"変更後(After) : "/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/02_tfrecord/tf_label_map.pbtxt"* 189行目(Line 189)：バリデーションデータの格納先(input_path)変更前(Before) : "PATH_TO_BE_CONFIGURED/val2017-?????-of-00032.tfrecord"変更後(After) : "/content/models/research/val_data/??????.tfrecord"![](https://user-images.githubusercontent.com/37477845/94040113-83558b80-fe03-11ea-8d8a-a5304efcca1d.png)  モデル訓練(Model training)  Googleドライブに保存先ディレクトリを作成(Create directory in Google Drive)<jupyter_code>!mkdir '/content/gdrive/My Drive/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On'<jupyter_output><empty_output><jupyter_text>TensorBoard<jupyter_code>%load_ext tensorboard
tensorboard --logdir '/content/gdrive/My Drive/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On'<jupyter_output><empty_output><jupyter_text>学習(Training)学習はColaboratory上で1000ステップにつき、約25分かかります(Learning takes about 25 minutes per 1000 steps on the Colaboratory)<jupyter_code>!python object_detection/model_main_tf2.py \
    --pipeline_config_path="/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/03_pretrained_model/pipeline.config" \
    --model_dir="/content/gdrive/My Drive/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On" \
    --num_train_steps=1000 \
    --alsologtostderr \
    --sample_1_of_n_eval_examples=1 \
    --num_eval_steps=100<jupyter_output><empty_output><jupyter_text>saved model形式へエクスポート(Export to saved-model format)<jupyter_code>!python object_detection/exporter_main_v2.py \
    --input_type=image_tensor \
    --pipeline_config_path="/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/03_pretrained_model/pipeline.config" \
    --trained_checkpoint_dir="/content/gdrive/My Drive/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On" \
    --output_directory="/content/gdrive/My Drive/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/output"<jupyter_output><empty_output><jupyter_text>モデルロード(Load model)<jupyter_code>import tensorflow as tf

model_path = '/content/gdrive/My Drive/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/output/saved_model'

DEFAULT_FUNCTION_KEY = 'serving_default'
loaded_model = tf.saved_model.load(model_path)
inference_func = loaded_model.signatures[DEFAULT_FUNCTION_KEY]<jupyter_output><empty_output><jupyter_text>推論(Inference)<jupyter_code># 推論対象のテスト画像一覧(List of test images to be inferred)
import glob
import copy
import cv2
from google.colab.patches import cv2_imshow

test_data_dir = '/content/models/research/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On/04_test_data'
testfile_list = sorted(glob.glob(test_data_dir + '/*.jpg'))
# 推論用関数(Function for inference)
def run_inference_single_image(image, inference_func):
    tensor = tf.convert_to_tensor(image)
    output = inference_func(tensor)

    output['num_detections'] = int(output['num_detections'][0])
    output['detection_classes'] = output['detection_classes'][0].numpy()
    output['detection_boxes'] = output['detection_boxes'][0].numpy()
    output['detection_scores'] = output['detection_scores'][0].numpy()
    return output
import cv2
import numpy as np
from PIL import Image

filenames = []

# 動画書き出し用設定(VideoWriter setting)
temp_image = cv2.imread(testfile_list[0], cv2.IMREAD_UNCHANGED)
image_width, image_height = temp_image.shape[1], temp_image.shape[0]
fourcc = 'mp4v'
writer_fourcc = cv2.VideoWriter_fourcc(*fourcc)
videowriter = cv2.VideoWriter('result.mp4', writer_fourcc, 10, (image_width, image_height))

# 推論(Inference)
for filecount, testfile in enumerate(testfile_list):
    image = cv2.imread(testfile, cv2.IMREAD_UNCHANGED)
    debug_image = copy.deepcopy(image)

    image_width, image_height = image.shape[1], image.shape[0]
    image = image[:, :, [2, 1, 0]]  # BGR2RGB
    image_np_expanded = np.expand_dims(image, axis=0)

    output = run_inference_single_image(image_np_expanded, inference_func)

    num_detections = output['num_detections']
    for i in range(num_detections):
        score = output['detection_scores'][i]
        bbox = output['detection_boxes'][i]
        # class_id = output['detection_classes'][i].astype(np.int)

        if score < 0.85:
            continue

        x1, y1 = int(bbox[1] * image_width), int(bbox[0] * image_height)
        x2, y2 = int(bbox[3] * image_width), int(bbox[2] * image_height)

        # 推論結果描画(Inference result drawing)
        cv2.rectangle(debug_image, (x1, y1), (x2, y2), (255, 255, 255), 2)
        cv2.putText(debug_image, str('{:.2f}'.format(score)), (x1, y1-10), cv2.FONT_HERSHEY_PLAIN, 1.5, (255, 255, 255), 2, cv2.LINE_AA)
        cv2.rectangle(debug_image, (x1, y1), (x2, y2), (255, 255, 255), 2)
    videowriter.write(debug_image)
videowriter.release()<jupyter_output><empty_output><jupyter_text>推論結果確認(Inference result confirmation)<jupyter_code>import imageio
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from IPython.display import HTML

def play_video(video, interval=100):
    video = imageio.mimread(video)
    fig = plt.figure(figsize=(9, 6))

    movie = []
    for i in range(len(video)):
        img = plt.imshow(video[i], animated=True)
        plt.axis('off')
        movie.append([img])

    anime = animation.ArtistAnimation(fig, movie, interval=interval, repeat_delay=1000)
    plt.close()
    return anime

HTML(play_video('result.mp4').to_html5_video())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12260500235960359, 'fraction_numerical': 0.036715431807456345, 'mean_word_length': 6.590257879656161, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 49, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '661440', 'license': 'MIT', 'n_tokens_mistral': 4471, 'n_tokens_neox': 4204, 'n_words': 509, 'path': '[Colaboratory]Tensorflow2-ObjectDetectionAPI-Colab-Hands-On.ipynb', 'repo_name': 'gizzytom/Tensorflow2-ObjectDetectionAPI-Colab-Hands-On', 'size': '10595'}