<jupyter_start><jupyter_text>Load data<jupyter_code>import tensorflow as tf

mnist = tf.keras.datasets.mnist

(X_train, y_train), (X_test, y_test) = mnist.load_data()

# feature scaling (data normalisation)
# Prediction accuracy gets higher with scaled images.
# Original colour vector range 0 ~ 255  ---> scaled range 0.0 ~ 1.0 
X_train, X_test = X_train/255.0, X_test/255.0<jupyter_output><empty_output><jupyter_text>Multiple ways to create a model   1) Using Keras<jupyter_code>mlp_model = tf.keras.models.Sequential([
                                        tf.keras.layers.Flatten(input_shape=(28, 28)),
                                        tf.keras.layers.Dense(128, activation='relu'),
                                        tf.keras.layers.Dense(10, activation='softmax')
])

mlp_model.fit(X_train, y_train, epochs=5)
mlp_model.evaluate(X_test, y_test, verbose=1)<jupyter_output><empty_output><jupyter_text>2) Using functional API<jupyter_code># DNN - multi-layer perceptrons
inputs = tf.keras.Input(shape=(28,28))
x = tf.keras.layers.Flatten()(inputs)
x = tf.keras.layers.Dense(128, activation='relu')(x)
outputs = tf.keras.layers.Dense(10, activation='softmax')(x)

mlp_model = tf.keras.Model(inputs=inputs, outputs=outputs)

mlp_model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',
                  metrics=['accuracy'])

mlp_model.fit(X_train, y_train, epochs=5)
mlp_model.evaluate(X_test, y_test, verbose=1)<jupyter_output>Epoch 1/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.2637 - accuracy: 0.9234
Epoch 2/5
1875/1875 [==============================] - 3s 1ms/step - loss: 0.1172 - accuracy: 0.9653
Epoch 3/5
1875/1875 [==============================] - 3s 1ms/step - loss: 0.0798 - accuracy: 0.9756
Epoch 4/5
1875/1875 [==============================] - 3s 1ms/step - loss: 0.0602 - accuracy: 0.9816
Epoch 5/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0455 - accuracy: 0.9856
313/313 [==============================] - 0s 1ms/step - loss: 0.0772 - accuracy: 0.9771<jupyter_text>3) Using class (best format)- Highest re-usability<jupyter_code># 재사용성이 가장 높음 - 모델 자체를 배포할 때도 이 방식 사용

class MLP_Model(tf.keras.Model):
    def __init__(self):
        super(MLP_Model, self).__init__()
        self.flatten = tf.keras.layers.Flatten()
        self.dense = tf.keras.layers.Dense(128, activation='relu')
        self.softmax = tf.keras.layers.Dense(10, activation='softmax')

    def call(self, inputs):
        x = self.flatten(inputs)
        x = self.dense(x)
        return self.softmax(x)

mlp_model = MLP_Model()

mlp_model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',
                  metrics=['accuracy'])

mlp_model.fit(X_train, y_train, epochs=5)
mlp_model.evaluate(X_test, y_test, verbose=2)<jupyter_output>Epoch 1/5
1875/1875 [==============================] - 4s 2ms/step - loss: 0.2585 - accuracy: 0.9266
Epoch 2/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.1139 - accuracy: 0.9666
Epoch 3/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0786 - accuracy: 0.9763
Epoch 4/5
1875/1875 [==============================] - 4s 2ms/step - loss: 0.0583 - accuracy: 0.9820
Epoch 5/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0456 - accuracy: 0.9855
313/313 - 0s - loss: 0.0769 - accuracy: 0.9759<jupyter_text>4) CNN<jupyter_code>import tensorflow as tf

mnist = tf.keras.datasets.mnist

(X_train, y_train), (X_test, y_test) = mnist.load_data()

X_train, X_test = X_train/255.0, X_test/255.0

# 입력 데이터 수정
X_train_4d = X_train.reshape(-1, 28, 28, 1)
X_test_4d = X_test.reshape(-1, 28, 28, 1)

cnn_model = tf.keras.models.Sequential([
            tf.keras.layers.Conv2D(32, (3,3), activation='relu',
                                   input_shape=(28,28,1)),
            tf.keras.layers.MaxPooling2D((2,2)),                     # MaxPooling merges neurons.
            tf.keras.layers.Conv2D(64, (3,3), activation='relu'),    # So increase the neuron quantity the next layer.
            tf.keras.layers.MaxPooling2D((2,2)),
            tf.keras.layers.Conv2D(64, (3,3), activation='relu'),
            tf.keras.layers.Flatten(),
            tf.keras.layers.Dense(64, activation='relu'),
            tf.keras.layers.Dense(10, activation='softmax')
])

cnn_model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',
                  metrics=['accuracy'])

cnn_model.fit(X_train_4d, y_train, epochs=5)

cnn_model.evaluate(X_test_4d, y_test, verbose=1)<jupyter_output>Epoch 1/5
1875/1875 [==============================] - 14s 7ms/step - loss: 0.1360 - accuracy: 0.9581
Epoch 2/5
1875/1875 [==============================] - 14s 8ms/step - loss: 0.0461 - accuracy: 0.9855
Epoch 3/5
1875/1875 [==============================] - 14s 8ms/step - loss: 0.0333 - accuracy: 0.9895
Epoch 4/5
1875/1875 [==============================] - 14s 8ms/step - loss: 0.0261 - accuracy: 0.9917
Epoch 5/5
1875/1875 [==============================] - 14s 8ms/step - loss: 0.0186 - accuracy: 0.9937
313/313 [==============================] - 1s 2ms/step - loss: 0.0321 - accuracy: 0.9909<jupyter_text>5) ResNet<jupyter_code>import tensorflow as tf

# Load and scale data.
mnist = tf.keras.datasets.mnist
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train, X_test = X_train/255.0, X_test/255.0

# Change the input data shape. 
X_train_4d = X_train.reshape(-1, 28, 28, 1)
X_test_4d = X_test.reshape(-1, 28, 28, 1)

# Further modify the shape for ResNet
resized_X_train = tf.image.resize(X_train_4d, [32, 32])
resized_X_test = tf.image.resize(X_test_4d, [32, 32])

# Create model
resnet_model = tf.keras.applications.ResNet50V2(
                    input_shape=(32,32,1),
                    classes=10,
                    weights=None
)

resnet_model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',
                  metrics=['accuracy'])

resnet_model.fit(resized_X_train, y_train, epochs=5)

resnet_model.evaluate(resized_X_test, y_test, verbose=1)

# ResNet is deeper than other models created here. So it takes longer.<jupyter_output>Epoch 1/5
1875/1875 [==============================] - 410s 216ms/step - loss: 0.2244 - accuracy: 0.9387
Epoch 2/5
1875/1875 [==============================] - 402s 215ms/step - loss: 0.1244 - accuracy: 0.9684
Epoch 3/5
1875/1875 [==============================] - 421s 224ms/step - loss: 0.0755 - accuracy: 0.9787
Epoch 4/5
1875/1875 [==============================] - 428s 228ms/step - loss: 0.0866 - accuracy: 0.9780
Epoch 5/5
1875/1875 [==============================] - 432s 231ms/step - loss: 0.0656 - accuracy: 0.9824
313/313 [==============================] - 18s 55ms/step - loss: 0.0261 - accuracy: 0.9922<jupyter_text>Running the model  i. Model structure<jupyter_code>mlp_model.summary()<jupyter_output>Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 28, 28)]          0         
_________________________________________________________________
flatten (Flatten)            (None, 784)               0         
_________________________________________________________________
dense (Dense)                (None, 128)               100480    
_________________________________________________________________
dense_1 (Dense)              (None, 10)                1290      
=================================================================
Total params: 101,770
Trainable params: 101,770
Non-trainable params: 0
_________________________________________________________________<jupyter_text>ii. Compiling the modelOptions for loss fucntion: - Regression: `MSE` (value)   - or... `MAE`, `RMSE` are also available. - Classification: `CrossEntropy` (probability)   - binary classification: `binary_crossentropy`   - multi-classification: `sparse_categorical crossentropy`, `categorical_crossentropy`- One-hot encoded: `categorical_crossentropy`- None one-hot encoded: `sparse_categorical_crossentropy`<jupyter_code>y_train = tf.keras.utils.to_categorical(y_train)
y_test = tf.keras.utils.to_categorical(y_test)

mlp_model.compile(optimizer='adam',
                  loss='categorical_crossentropy',
                  metrics=['accuracy'])<jupyter_output>Epoch 1/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0377 - accuracy: 0.9881
Epoch 2/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0282 - accuracy: 0.9913
Epoch 3/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0231 - accuracy: 0.9927
Epoch 4/5
1875/1875 [==============================] - 3s 1ms/step - loss: 0.0189 - accuracy: 0.9942
Epoch 5/5
1875/1875 [==============================] - 3s 1ms/step - loss: 0.0160 - accuracy: 0.9949
313/313 - 0s - loss: 0.0746 - accuracy: 0.9789<jupyter_text>iii. Train the model<jupyter_code>mlp_model.fit(X_train, y_train, epochs=5)<jupyter_output>Epoch 1/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0129 - accuracy: 0.9963
Epoch 2/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0104 - accuracy: 0.9968
Epoch 3/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0105 - accuracy: 0.9969
Epoch 4/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0083 - accuracy: 0.9975
Epoch 5/5
1875/1875 [==============================] - 3s 1ms/step - loss: 0.0081 - accuracy: 0.9974<jupyter_text>iv. Evaluate the model<jupyter_code>mlp_model.evaluate(X_test, y_test, verbose=1)<jupyter_output>313/313 [==============================] - 0s 972us/step - loss: 0.0991 - accuracy: 0.9778<jupyter_text>6) Create a transfer-learning model - Import from *Mobile TensorFlow Hub*.   - https://tfhub.dev/- Convert to Tensorflow Light model (ext .tflite)Process: 1) Create a converter:    - `tensorflow.lite.TFLiteConverter.from_keras_model(model_name)`  2) Write it to a file.    - `with open('/content/here/MyDrive/MLP_model/keras_model.tflite', 'wb') as f:`    - `f.write(tflite_model)`    -extension: `.tflite`                    model = tf.keras.Sequential([            tensorflow_hun.KerasLayer(url, input_shape=(입력 구조), trainable=False),            tf.keras.layers.Dense(출력의 개수)        ])  Convert the above models into a TensorFlow Lite model<jupyter_code>from google.colab import drive
drive.mount('here')
converter = tf.lite.TFLiteConverter.from_keras_model(mlp_model)
tflite_model = converter.convert()

with open('/content/here/MyDrive/MLP_model/keras_model.tflite', 'wb') as f:
    f.write(tflite_model)<jupyter_output>INFO:tensorflow:Assets written to: /tmp/tmpww6fw5n4/assets
{'fraction_non_alphanumeric': 0.2243974512882076, 'fraction_numerical': 0.09160587311847816, 'mean_word_length': 3.7107438016528924, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 37, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '498787', 'license': 'CC0-1.0', 'n_tokens_mistral': 4458, 'n_tokens_neox': 3751, 'n_words': 742, 'path': 'Multiple ways to create the same model.ipynb', 'repo_name': 'Coding-Forest/MachineLearning', 'size': '10829'}