<jupyter_start><jupyter_text>If we want to use our machine learning model in production or reuse our trained model for a transfer learning task, we have to store our model. In this section, we will outline some methods for storing and restoring the weights or the whole model.  Getting ready... In this recipe, we want to summarize various ways to store a TensorFlow model. We will cover the best way to save and restore an entire model, only the weights, and model checkpoints.  How to do it... 1. Load our libraries<jupyter_code>import tensorflow as tf<jupyter_output><empty_output><jupyter_text>2. Build an MNIST model using the Keras Sequential API<jupyter_code>(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

# Normalize
x_train = x_train / 255
y_train = y_train / 255

model = tf.keras.Sequential()
model.add(tf.keras.layers.Flatten(name='FLATTEN'))
model.add(tf.keras.layers.Dense(units=128, activation='relu', name='D1'))
model.add(tf.keras.layers.Dense(units=64, activation='relu', name='D2'))
model.add(tf.keras.layers.Dense(units=10, activation='softmax', name='OUTPUT'))

model.compile(optimizer='sgd',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x=x_train,
          y=y_train,
          epochs=5,
          validation_data=(x_test, y_test)
         )<jupyter_output>Epoch 1/5
1875/1875 [==============================] - 2s 895us/step - loss: 0.0148 - accuracy: 0.0986 - val_loss: 2990.2285 - val_accuracy: 0.0980
Epoch 2/5
1875/1875 [==============================] - 2s 831us/step - loss: 3.8037e-04 - accuracy: 0.0987 - val_loss: 3293.5259 - val_accuracy: 0.0980
Epoch 3/5
1875/1875 [==============================] - 2s 830us/step - loss: 2.1202e-04 - accuracy: 0.0987 - val_loss: 3473.6875 - val_accuracy: 0.0980
Epoch 4/5
1875/1875 [==============================] - 2s 822us/step - loss: 1.4631e-04 - accuracy: 0.0987 - val_loss: 3602.6611 - val_accuracy: 0.0980
Epoch 5/5
1875/1875 [==============================] - 2s 822us/step - loss: 1.1130e-04 - accuracy: 0.0987 - val_loss: 3703.5757 - val_accuracy: 0.0980<jupyter_text>3. We can now save our entire model on disk<jupyter_code>model.save("SavedModel")<jupyter_output>WARNING:absl:Function `_wrapped_model` contains input name(s) FLATTEN_input with unsupported characters which will be renamed to flatten_input in the SavedModel.<jupyter_text>4. A directory named SavedModel is created on disk. It contains a TensorFlow program, the saved_model.pb file; the variables directory, which contains the exact value of all parameters; and the assets directory, which contains files used by the TensorFlow graph:<jupyter_code>ls
cd SavedModel
ls<jupyter_output>[0m[01;34massets[0m/  keras_metadata.pb  saved_model.pb  [01;34mvariables[0m/<jupyter_text>5. We can now restore our saved model<jupyter_code>cd ..
model2 = tf.keras.models.load_model("SavedModel")<jupyter_output><empty_output><jupyter_text>6. If we prefer, we can save the model in the H5 format. We can either pass a file name that ends in .h5 or add the save_format='h5' argument<jupyter_code>model.save("SavedModel.h5")
model.save("model_save", save_format='h5')<jupyter_output><empty_output><jupyter_text>7. It is also possible to use the ModelCheckpoint callback in order to save an entire model or just the weights into a checkpoint structure at some intervals. This callback is added to the callback argument in the fit method. In the configuration below, the model weights will be stored at each epoch:<jupyter_code>checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(filepath='./checkpoint', save_weights_only=True, 
                                                         save_freq='epoch')

model.fit(x=x_train,
          y=y_train,
          epochs=5,
          validation_data=(x_test, y_test),
          callbacks=[checkpoint_callback])<jupyter_output>Epoch 1/5
1875/1875 [==============================] - 2s 845us/step - loss: 7.4869e-05 - accuracy: 0.0987 - val_loss: 3856.7004 - val_accuracy: 0.0980
Epoch 2/5
1875/1875 [==============================] - 2s 843us/step - loss: 6.4220e-05 - accuracy: 0.0987 - val_loss: 3917.8972 - val_accuracy: 0.0980
Epoch 3/5
1875/1875 [==============================] - 2s 840us/step - loss: 5.6172e-05 - accuracy: 0.0987 - val_loss: 3972.0359 - val_accuracy: 0.0980
Epoch 4/5
1875/1875 [==============================] - 2s 838us/step - loss: 4.9880e-05 - accuracy: 0.0987 - val_loss: 4020.6265 - val_accuracy: 0.0980
Epoch 5/5
1875/1875 [==============================] - 2s 832us/step - loss: 4.4829e-05 - accuracy: 0.0987 - val_loss: 4064.6948 - val_accuracy: 0.0980<jupyter_text>8. Now we can load the entire model or just the weights later in order to continue the training. Here, we are reloading the weights:<jupyter_code>model.load_weights('./checkpoint')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16400164001640016, 'fraction_numerical': 0.08815088150881509, 'mean_word_length': 4.403100775193798, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '572190', 'license': 'MIT', 'n_tokens_mistral': 1912, 'n_tokens_neox': 1636, 'n_words': 455, 'path': 'Chapter 12 - Taking TensorFlow to Production/Saving and restoring a TensorFlow model.ipynb', 'repo_name': 'Young-Picasso/TensorFlow2.x_Cookbook', 'size': '4878'}