<jupyter_start><jupyter_text>Customizing what happens in fit().When you need to write your own training loop from scratch, you can use the ``GradientTape`` and take control of every little detail.When you need to customize what ``fit()`` does, you should override the training step function of the Model class. This is the function that is called by ``fit()`` for every batch of data. You will then be able to call ``fit()`` as usual -- and it will be running your own learning algorithm.* [Docs](https://keras.io/guides/customizing_what_happens_in_fit/) Imports<jupyter_code>import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import datasets
import numpy as np<jupyter_output><empty_output><jupyter_text>Configuring the ``device`` for the environment.<jupyter_code>physical_devices = tf.config.list_physical_devices("GPU")
tf.config.experimental.set_memory_growth(physical_devices[0], True)<jupyter_output><empty_output><jupyter_text>Let's create a model that will train on the `MNIST` dataset.<jupyter_code>(X_train, y_train), (X_test, y_test) = datasets.mnist.load_data()
X_train.shape
def normalize(image):
  image = tf.convert_to_tensor(image.astype('float32'))/255
  return image
X_train_tensors =tf.convert_to_tensor(list(map(normalize, X_train)))
X_test_tensors = tf.convert_to_tensor(list(map(normalize, X_test)))

y_test_tensors = tf.convert_to_tensor(y_test)
y_train_tensors = tf.convert_to_tensor(y_train)
y_test_tensors[:2]<jupyter_output><empty_output><jupyter_text>Let's create a `Sequential` model that will fit on our custom `.fit()` method. We can do this in many ways. [2nd way](https://keras.io/guides/customizing_what_happens_in_fit/)<jupyter_code>model = keras.Sequential([
      keras.layers.Input(shape=(28, 28,)),
      keras.layers.Flatten(),
      keras.layers.Dense(64, activation="relu"),
      keras.layers.Dense(128, activation="relu"),
      keras.layers.Dense(10, activation="softmax")
])
class CustomFit(keras.Model):
   def __init__(self, model):
    super(CustomFit, self).__init__()
    self.model = model
   def train_step(self, data):
        x, y = data
        with tf.GradientTape() as tape:
            # forward pass
            y_pred = self.model(x, training=True)
            # calclate the loss
            loss = self.compiled_loss(y, y_pred, regularization_losses=self.losses)

        # gradient calculations
        gradients = tape.gradient(loss, self.trainable_variables)
        # Update the weights
        self.optimizer.apply_gradients(zip(gradients, self.trainable_variables))
        self.compiled_metrics.update_state(y, y_pred)

        return {m.name: m.result() for m in self.metrics}<jupyter_output><empty_output><jupyter_text>Fitting the model using our custom ``fit``.<jupyter_code>new_model = CustomFit(model)
new_model.compile(loss=keras.losses.SparseCategoricalCrossentropy(),
                optimizer="adam",
                metrics=["accuracy"]
              )
new_model.fit(X_train_tensors, y_train_tensors, batch_size=32, epochs=5)<jupyter_output>Epoch 1/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0221 - accuracy: 0.9922
Epoch 2/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0175 - accuracy: 0.9938
Epoch 3/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0175 - accuracy: 0.9941
Epoch 4/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0144 - accuracy: 0.9951
Epoch 5/5
1875/1875 [==============================] - 3s 2ms/step - loss: 0.0151 - accuracy: 0.9947<jupyter_text>Now let's create a custom `.fit()` and `.evaluate()`.We have seen thet to create a `.fit()` method we need to overide the `train_step` which mean if we want to create the `.evaluate()` we need to overide the `test_step`.<jupyter_code>class CustomFitEval(keras.Model):
  def __init__(self, model):
    super().__init__()
    self.model = model

  # fit
  def train_step(self, data):
    x, y = data
    # forward pass
    with tf.GradientTape() as tape:
      y_pred = self.model(x, training=True)
      # loss
      loss = self.compiled_loss(y, y_pred)
    # caculate the gradients
    gradients = tape.gradient(loss, self.trainable_variables)
    # update the weights
    self.optimizer.apply_gradients(zip(gradients, self.trainable_variables))
      
    # update the metrics
    self.compiled_metrics.update_state(y, y_pred)
    return {m.name: m.result() for m in self.metrics}

  # evaluate
  def test_step(self, data):
    x, y = data
    # making predictions
    y_pred = self.model(x)
    # loss
    loss = self.compiled_loss(y, y_pred)
    self.compiled_metrics.update_state(y, y_pred)
    return {m.name: m.result() for m in self.metrics}
model = keras.Sequential([
      keras.layers.Input(shape=(28, 28,)),
      keras.layers.Flatten(),
      keras.layers.Dense(64, activation="relu"),
      keras.layers.Dense(128, activation="relu"),
      keras.layers.Dense(10, activation="softmax")
])
new_model_1 = CustomFitEval(model)

new_model_1.compile(loss=keras.losses.SparseCategoricalCrossentropy(),
                optimizer="adam",
                metrics=["accuracy"]
              )
new_model_1.fit(X_train_tensors, y_train_tensors, batch_size=32, epochs=5,
                validation_data=(X_test_tensors, y_test_tensors), validation_batch_size=16
                )<jupyter_output>WARNING:tensorflow:Please add `keras.layers.InputLayer` instead of `keras.Input` to Sequential model. `keras.Input` is intended to be used by Functional model.
Epoch 1/5
1875/1875 [==============================] - 4s 2ms/step - loss: 0.2610 - accuracy: 0.9236 - val_loss: 0.1215 - val_accuracy: 0.9629
Epoch 2/5
1875/1875 [==============================] - 4s 2ms/step - loss: 0.1124 - accuracy: 0.9649 - val_loss: 0.0986 - val_accuracy: 0.9709
Epoch 3/5
1875/1875 [==============================] - 4s 2ms/step - loss: 0.0835 - accuracy: 0.9740 - val_loss: 0.0949 - val_accuracy: 0.9701
Epoch 4/5
1875/1875 [==============================] - 4s 2ms/step - loss: 0.0660 - accuracy: 0.9790 - val_loss: 0.0903 - val_accuracy: 0.9718
Epoch 5/5
1875/1875 [==============================] - 4s 2ms/step - loss: 0.0552 - accuracy: 0.9821 - val_loss: 0.0813 - val_accuracy: 0.9761<jupyter_text>Evaluating the `model`<jupyter_code>new_model_1.evaluate(X_test_tensors, y_test_tensors, batch_size=32)<jupyter_output>313/313 [==============================] - 1s 2ms/step - loss: 0.0886 - accuracy: 0.9736
{'fraction_non_alphanumeric': 0.16625348621010227, 'fraction_numerical': 0.051440966842268364, 'mean_word_length': 4.135242641209229, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '603476', 'license': 'MIT', 'n_tokens_mistral': 2432, 'n_tokens_neox': 2156, 'n_words': 500, 'path': '11_Custom_Model_Fit_and_Training_Loops/01_Custom_Model_fit.ipynb', 'repo_name': 'CrispenGari/keras-api', 'size': '6454'}