<jupyter_start><jupyter_text>Create models for TensorFlow Lite for Microcontrollers
- Create the original model using Keras (conv_mnist.h5)
- Convert the model into TensorFlow Lite
  - Without quantization (conv_mnist.tflite)
  - With quantization (conv_mnist_quant.tflite)
- Convert tflite model into C code using xxd  Create the original model using Keras (conv_mnist.h5)
Simple MNIST model<jupyter_code>%tensorflow_version 2.x
from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
import numpy as np

print(tf.__version__)

# MNISTの学習用データ、テストデータをロードする
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

# 学習データの前処理
# X: 6000x28x28x1のTensorに変換し、値を-1.0～1.0に正規化
# Y: one-hot化(6000x1 -> 6000x10)   -> no need when using SparseCategoricalCrossentropy()
x_train = x_train.reshape(x_train.shape[0], x_train.shape[1], x_train.shape[2], 1)
x_test = x_test.reshape(x_test.shape[0], x_test.shape[1], x_test.shape[2], 1)
x_train = x_train / 128. - 1.
x_test = x_test / 128. - 1.
x_train = x_train.astype(np.float32)
x_test = x_test.astype(np.float32)
# y_train = tf.keras.utils.to_categorical(y_train, 10)
# y_test = tf.keras.utils.to_categorical(y_test, 10)

# 学習状態把握用のTensorBoard設定
tsb = tf.keras.callbacks.TensorBoard(log_dir="./logs")

# Convolutionモデルの作成
input = tf.keras.layers.Input(shape=(28,28,1), name="input_image")
conv1 = tf.keras.layers.Conv2D(
  filters=8,
  kernel_size=(3,3),
  strides=(1,1),
  padding="same",
  activation="relu"
)(input)
pool1 = tf.keras.layers.MaxPooling2D(pool_size=(2,2))(conv1)
conv2 = tf.keras.layers.Conv2D(
  filters=4,
  kernel_size=(3,3),
  strides=(1,1),
  padding="same",
  activation="relu"
)(pool1)
dropout1 = tf.keras.layers.Dropout(0.2)(conv2)
flatten1 = tf.keras.layers.Flatten()(dropout1)
output = tf.keras.layers.Dense(units=10, activation="softmax", name="output_scores")(flatten1)
model = tf.keras.models.Model(inputs=[input], outputs=[output], name="ConvMnist")
model.summary()

model.compile(
  optimizer=tf.keras.optimizers.Adam(),
  loss=tf.keras.losses.SparseCategoricalCrossentropy(),
  metrics=[tf.keras.metrics.SparseCategoricalAccuracy()]
)

# Convolutionモデルの学習
model.fit(
  x_train,
  y_train,
  batch_size=32,
  epochs=20,
  validation_split=0.2,
  callbacks=[tsb],
)

# 学習したモデルを使用して、テスト用データで評価する
model.evaluate(x_test,  y_test, verbose=2)

# 学習済みモデルをファイル(h5)に保存する
model.save("conv_mnist.h5")<jupyter_output>2.4.1
Model: "ConvMnist"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_image (InputLayer)     [(None, 28, 28, 1)]       0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 28, 28, 8)         80        
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 14, 14, 8)         0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 14, 14, 4)         292       
_________________________________________________________________
dropout_1 (Dropout)          (None, 14, 14, 4)         0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 784)               0         
___________________________________________________[...]<jupyter_text>Test the model<jupyter_code>import cv2
from google.colab.patches import cv2_imshow
import numpy as np

# Read input image
!wget -O 4.jpg  "https://drive.google.com/uc?export=download&id=1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A" 
img_org = cv2.imread("4.jpg")
cv2_imshow(img_org)

# Pre process
## グレースケール化、リサイズ、白黒判定、価範囲を0～255 -> -1.0～1.0
img = cv2.cvtColor(img_org, cv2.COLOR_BGR2GRAY)
img = cv2.resize(img, (28, 28))
img = 255 - img
img = img / 128. - 1.
img = img.astype(np.float32)
input_tensor = img.reshape(1, img.shape[0], img.shape[1], 1)

# Load model
loaded_model = tf.keras.models.load_model("conv_mnist.h5")

# Inference
scores = loaded_model.predict(input_tensor)

result = np.argmax(scores[0])
print("predicted number is {} [{:.2f}]".format(result, scores[0][result]))<jupyter_output>--2021-02-14 16:54:53--  https://drive.google.com/uc?export=download&id=1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A
Resolving drive.google.com (drive.google.com)... 192.168.3.11, 192.168.3.11, 172.16.31.10, ...
Connecting to drive.google.com (drive.google.com)|192.168.3.11|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://doc-0c-7s-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/4qub1roodu7jv7isu3jjqag0iu96gcuh/1613321625000/06137305555544994122/*/1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A?e=download [following]
Warning: wildcards not supported in HTTP.
--2021-02-14 16:54:53--  https://doc-0c-7s-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/4qub1roodu7jv7isu3jjqag0iu96gcuh/1613321625000/06137305555544994122/*/1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A?e=download
Resolving doc-0c-7s-docs.googleusercontent.com (doc-0c-7s-docs.googleusercontent.com)... 192.168.3.11, fdf8:f53e:61e4::18
Connecting to do[...]<jupyter_text>Convert the model into TensorFlow Lite without quantization (conv_mnist.tflite)<jupyter_code>%tensorflow_version 2.x
from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf

# Convert
loaded_model = tf.keras.models.load_model("conv_mnist.h5")
converter = tf.lite.TFLiteConverter.from_keras_model(loaded_model)

tflite_model = converter.convert()
open("conv_mnist.tflite", "wb").write(tflite_model)<jupyter_output>INFO:tensorflow:Assets written to: /tmp/tmpojf78kbl/assets<jupyter_text>Test the model<jupyter_code>import cv2
from google.colab.patches import cv2_imshow
import numpy as np

# Read input image
!wget -O 4.jpg  "https://drive.google.com/uc?export=download&id=1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A" 
img = cv2.imread("4.jpg")
cv2_imshow(img)

# Pre process
## グレースケール化、リサイズ、白黒判定、価範囲を0～255 -> -1.0～1.0
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img = cv2.resize(img, (28, 28))
img = 255 - img
img = img / 128. - 1.
img = img.astype(np.float32)
input_tensor = img.reshape(1, img.shape[0], img.shape[1], 1)
input_tensor = tf.convert_to_tensor(input_tensor)

# Load model
interpreter = tf.lite.Interpreter(model_path="conv_mnist.tflite")
# interpreter = tf.lite.Interpreter(model_path="conv_mnist_from_pb.tflite")
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# set input tensor
interpreter.set_tensor(input_details[0]['index'], input_tensor)

# Inference
interpreter.invoke()

scores = interpreter.get_tensor(output_details[0]['index'])
result = np.argmax(scores[0])
print("predicted number is {} [{:.2f}]".format(result, scores[0][result]))<jupyter_output>--2021-02-14 16:55:15--  https://drive.google.com/uc?export=download&id=1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A
Resolving drive.google.com (drive.google.com)... 192.168.3.11, 192.168.3.11, 172.16.31.10, ...
Connecting to drive.google.com (drive.google.com)|192.168.3.11|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://doc-0c-7s-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/bu9bs8o00f7iqhc6ai42cusdc7tjthgd/1613321700000/06137305555544994122/*/1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A?e=download [following]
Warning: wildcards not supported in HTTP.
--2021-02-14 16:55:15--  https://doc-0c-7s-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/bu9bs8o00f7iqhc6ai42cusdc7tjthgd/1613321700000/06137305555544994122/*/1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A?e=download
Resolving doc-0c-7s-docs.googleusercontent.com (doc-0c-7s-docs.googleusercontent.com)... 192.168.3.11, fdf8:f53e:61e4::18
Connecting to do[...]<jupyter_text>Convert the model into TensorFlow Lite with quantization (conv_mnist_quant.tflite)<jupyter_code>%tensorflow_version 2.x
from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
import numpy as np

print(tf.__version__)

## Prepara dataset generator for calibration
(x_train, _), (_, _) = tf.keras.datasets.mnist.load_data()
x_train = x_train.reshape(x_train.shape[0], x_train.shape[1], x_train.shape[2], 1)
x_train = x_train / 128.0 - 1
x_train = x_train.astype(np.float32)
num_calibration_images = 100
calibration_indexes   = np.random.choice(x_train.shape[0], num_calibration_images, replace=False)
def representative_dataset_gen():
  for i in range(num_calibration_images):
    yield [x_train[calibration_indexes[i: i + 1]]]


# Convert
loaded_model = tf.keras.models.load_model("conv_mnist.h5")
converter = tf.lite.TFLiteConverter.from_keras_model(loaded_model)
# converter = tf.lite.TFLiteConverter.from_keras_model_file("conv_mnist.h5")

converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen

# For full integer quantization
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
## NOT UINT8 but INT8
# converter.inference_input_type = tf.uint8
# converter.inference_output_type = tf.uint8
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
converter.experimental_new_converter = True   # will be no need in the future

tflite_model = converter.convert()
open("conv_mnist_quant.tflite", "wb").write(tflite_model)<jupyter_output>2.4.1
INFO:tensorflow:Assets written to: /tmp/tmpu0yzrj3a/assets<jupyter_text>Test the model<jupyter_code>import cv2
from google.colab.patches import cv2_imshow
import numpy as np

# Read input image
!wget -O 4.jpg  "https://drive.google.com/uc?export=download&id=1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A" 
img = cv2.imread("4.jpg")
cv2_imshow(img)

# Pre process
## グレースケール化、リサイズ、白黒判定、価範囲を-128～127
img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
img = cv2.resize(img, (28, 28))
img = (255 - img) - 128

# img = img.astype(np.uint8)
img = img.astype(np.int8)
input_tensor = img.reshape(1, img.shape[0], img.shape[1], 1)
input_tensor = tf.convert_to_tensor(input_tensor)

# Load model
interpreter = tf.lite.Interpreter(model_path="conv_mnist_quant.tflite")
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# set input tensor
interpreter.set_tensor(input_details[0]['index'], input_tensor)

# Inference
interpreter.invoke()

scores = interpreter.get_tensor(output_details[0]['index'])
result = np.argmax(scores[0])

print("predicted number is {} [{:.2f}]".format(result, scores[0][result]))<jupyter_output>--2021-02-14 17:01:51--  https://drive.google.com/uc?export=download&id=1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A
Resolving drive.google.com (drive.google.com)... 172.16.17.32, 192.168.3.11, 192.168.127.12, ...
Connecting to drive.google.com (drive.google.com)|172.16.17.32|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://doc-0c-7s-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/ittbnpkc0m4894otjsel4c8i3d5scjj4/1613322075000/06137305555544994122/*/1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A?e=download [following]
Warning: wildcards not supported in HTTP.
--2021-02-14 17:01:51--  https://doc-0c-7s-docs.googleusercontent.com/docs/securesc/ha0ro937gcuc7l7deffksulhg5h7mbp1/ittbnpkc0m4894otjsel4c8i3d5scjj4/1613322075000/06137305555544994122/*/1-3yb3qCrN8M6Bdj7ZZ9UMjONh34R2W_A?e=download
Resolving doc-0c-7s-docs.googleusercontent.com (doc-0c-7s-docs.googleusercontent.com)... 192.168.3.11, fdf8:f53e:61e4::18
Connecting to do[...]<jupyter_text>Convert tflite model into C code using xxd<jupyter_code>!sudo apt install xxd
!xxd -i conv_mnist_quant.tflite > conv_mnist_quant.cpp
!xxd -i conv_mnist.tflite > conv_mnist.cpp
!ls -la
!mkdir conv_mnist
!cp conv_mnist*.* conv_mnist/.
!zip conv_mnist.zip -r conv_mnist
from google.colab import files
files.download( "./conv_mnist.zip")<jupyter_output>mkdir: cannot create directory ‘conv_mnist’: File exists
updating: conv_mnist/ (stored 0%)
updating: conv_mnist/conv_mnist_quant.cpp (deflated 81%)
updating: conv_mnist/conv_mnist.cpp (deflated 76%)
updating: conv_mnist/conv_mnist_quant.tflite (deflated 29%)
updating: conv_mnist/conv_mnist.h5 (deflated 29%)
updating: conv_mnist/conv_mnist.tflite (deflated 10%)
updating: conv_mnist/conv_mnist.zip (stored 0%)
{'fraction_non_alphanumeric': 0.1304176826870133, 'fraction_numerical': 0.0803901518131047, 'mean_word_length': 5.256889763779528, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 28, 'https://': 12, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 15, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 13, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '187214', 'license': 'Apache-2.0', 'n_tokens_mistral': 5664, 'n_tokens_neox': 5142, 'n_words': 746, 'path': 'pj_mnist/conv_mnist/tflmicro_mnist.ipynb', 'repo_name': 'iwatake2222/pico-mnist', 'size': '12753'}