<jupyter_start><jupyter_text>This is a companion notebook for the book [Deep Learning with Python, Second Edition](https://www.manning.com/books/deep-learning-with-python-second-edition?a_aid=keras&a_bid=76564dff). For readability, it only contains runnable code blocks and section titles, and omits everything else in the book: text paragraphs, figures, and pseudocode.**If you want to be able to follow what's going on, I recommend reading the notebook side by side with your copy of the book.**This notebook was generated for TensorFlow 2.6.  Working with Keras: A deep dive- The previous chapters used Sequential model,  dense layers, and built-in APIs for training, evaluation, and inference using, respectively, compile(), fit(), evaluate(), predict().- Complex applications in computer vision, time series forecasting, natural language processing, and deep generative models require more than sequential architecture and the default fit() loop.- This chapter will provide a complete overview of key ways of working with Keras API that would be needed for handling advanced deep learning use cases.  A spectrum of workflows- Keras API is designed on the principle of *progressive disclosure of complexity*. It is easy to get started on, but progressively goes on to handle complex use cases.- Instead of a single "true" way of using Keras, it offers a *spectrum of workflows*, and the compoenents from any workflow can be used in any other workflow.  Different ways to build Keras modelsThere are three APIs for building models in Keras.- The **Sequential model**, the most approachable API, is just a Python list, and is limited to simple stack of layers.- The **Functional API** focuses on graph-like architecture, and offers a nice middle grounds -- in terms of usability and flexibility. It is the most widely used API.- **Model subclassing** is a low level option where you write everything from stratch. The complete control comes at the cost of non-availability of built-in Keras features nad the risk fo making mistakes.  The Sequential modelThis is the simplest way to build a Keras model. **The `Sequential` class**We buid a model with two dense layers.<jupyter_code>from tensorflow import keras
from tensorflow.keras import layers

model = keras.Sequential([
    layers.Dense(64, activation="relu"),
    layers.Dense(10, activation="softmax")
])<jupyter_output><empty_output><jupyter_text>**Incrementally building a Sequential model**We could have also built this model incrementally by using the add() method, as shown below.<jupyter_code>model = keras.Sequential()
model.add(layers.Dense(64, activation="relu"))
model.add(layers.Dense(10, activation="softmax"))<jupyter_output><empty_output><jupyter_text>**Calling a model for the first time to build it**- The model has no data yet. So the shape of the weights matrices cannot be known at this time because it depends on shape of the input.- The layers will get built when you use the build() method and supply the input shape.- Below we specify input shape as ('None', 3) which means we have 3 input variables and the 'None' dimension signals that the batch-size could be anything.<jupyter_code>model.build(input_shape=(None, 3))    
model.weights

# Note: The 'None' argument in the input_shape means
# Note: Output below is deleted.<jupyter_output><empty_output><jupyter_text>**The summary method**After building the model, the summary() method will display a summary.<jupyter_code>model.summary()<jupyter_output>Model: "sequential_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_2 (Dense)             (None, 64)                256       
                                                                 
 dense_3 (Dense)             (None, 10)                650       
                                                                 
=================================================================
Total params: 906
Trainable params: 906
Non-trainable params: 0
_________________________________________________________________<jupyter_text>**Naming models and layers with the `name` argument**- Instead of letting Keras give default names such as 'Dense_1" to layers, both the  layer name and the model name can be user-specified. See below.<jupyter_code>model = keras.Sequential(name="my_example_model")
model.add(layers.Dense(64, activation="relu", name="my_first_layer"))
model.add(layers.Dense(10, activation="softmax", name="my_last_layer"))
model.build((None, 3))
model.summary()<jupyter_output>Model: "my_example_model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 my_first_layer (Dense)      (None, 64)                256       
                                                                 
 my_last_layer (Dense)       (None, 10)                650       
                                                                 
=================================================================
Total params: 906
Trainable params: 906
Non-trainable params: 0
_________________________________________________________________<jupyter_text>**Specifying the input shape of your model in advance**- The input shape can be specified upfront in the Keras  sequential model, using model.add(). This has the advantage that we can see the output shape changes as we add more layers to the sequential model. This can be done by using summary() after every layer.- The cell below specifies the input shape and a dense layer (with 64 units).- This would allow getting output shape after adding the first layer by using summary().- The subsequent cells will add more layers and get output shape using  summary() again.<jupyter_code>model = keras.Sequential()
model.add(keras.Input(shape=(3,)))
model.add(layers.Dense(64, activation="relu"))
# getting output shape with only the 1st layer
model.summary()
# adding 2nd layer (softmax), getting output shape
model.add(layers.Dense(10, activation="softmax"))
model.summary()<jupyter_output>Model: "sequential_2"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_4 (Dense)             (None, 64)                256       
                                                                 
 dense_5 (Dense)             (None, 10)                650       
                                                                 
=================================================================
Total params: 906
Trainable params: 906
Non-trainable params: 0
_________________________________________________________________<jupyter_text>The Functional API- The Sequential API is easy to use but it can handle only the models with one input and one output. Its applicability is limited.- Models may have multiple inputs (image data and its metadata) and multiple outputs (different things you want to predict about data) or a nonlinear topology.  A simple exampleThis example uses the functional API to implement the two-layer model from the previous section  **A simple Functional model with two `Dense` layers**- First, an 'inputs' object is created by a sybolic input tensor with shape (3,).- The inputs object is passed on to a dense layer (64 units, relu) that creates a features object.- The 'features' object is passed on to another dense layer (10 units, softmax) that creates an 'outputs' object<jupyter_code>inputs = keras.Input(shape=(3,), name="my_input")
features = layers.Dense(64, activation="relu")(inputs)
outputs = layers.Dense(10, activation="softmax")(features)
model = keras.Model(inputs=inputs, outputs=outputs)<jupyter_output><empty_output><jupyter_text>> Lets recreate the inputs, features, and outputs objects so that their shapes and dtype can be found.> In the end, model summary is generated.<jupyter_code>inputs = keras.Input(shape=(3,), name="my_input")
inputs.shape
inputs.dtype
features = layers.Dense(64, activation="relu")(inputs)
features.shape
outputs = layers.Dense(10, activation="softmax")(features)
model = keras.Model(inputs=inputs, outputs=outputs)
model.summary()<jupyter_output>Model: "model_1"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 my_input (InputLayer)       [(None, 3)]               0         
                                                                 
 dense_9 (Dense)             (None, 64)                256       
                                                                 
 dense_10 (Dense)            (None, 10)                650       
                                                                 
=================================================================
Total params: 906
Trainable params: 906
Non-trainable params: 0
_________________________________________________________________<jupyter_text>Multi-input, multi-output modelsThe functional API is best suited for multi-input, multi-output models. Use case: Building a system for ranking customers' tickets by priority and routing them to appropriate department.The model has three inputs- *The title of the ticket (text input)*- *The text body of the ticket (text input)*- *Any tags added by the user (categorical input, assumed to be one-hot encoded)*There are two outputs- *Priority score of the ticket, a scalar between 0 and 1 (sigmoid output)*- *The department that should handle the ticket (a softmax over the set of departments)*Below, this model will be built with a Functional API **A multi-input, multi-output Functional model**Preprocessing involves setting a vocabolary size of 10,000 words for textual data, setting the number of tags in the ticket text to 100, and setting the number of Departments to 4. The model definition involves the following steps:- Define model inputs- Combine input features into a single tensor, *features*, by concatenating them- Apply intermediate layer to recombine input features into richer representations (pass features to Dense layer)- Define model outputs- Create model by specifying its inputs and outputs<jupyter_code>vocabulary_size = 10000
num_tags = 100
num_departments = 4

# model inputs
title = keras.Input(shape=(vocabulary_size,), name="title")
text_body = keras.Input(shape=(vocabulary_size,), name="text_body")
tags = keras.Input(shape=(num_tags,), name="tags")

# recombining inputs into features 
features = layers.Concatenate()([title, text_body, tags])
features = layers.Dense(64, activation="relu")(features)

# model inputs:  ticket priority & the Department to which ticket is routed
priority = layers.Dense(1, activation="sigmoid", name="priority")(features)
department = layers.Dense(
    num_departments, activation="softmax", name="department")(features)

# creating the model
model = keras.Model(inputs=[title, text_body, tags], outputs=[priority, department])<jupyter_output><empty_output><jupyter_text>Training a multi-input, multi-output model **Training a model by providing lists of input & target arrays**The training process involves:- Initializing inputs as lists of random integers of dimensions (num_samples, vocabulary_size) with integers in (0,2) i.e as lists of 1's and 0's.- Initializing outputs- Compiling the model -specifying the multioutput  loss function with two loss metrics MSE and categorical cross entropy, corresponding respectively to outputs priority and department; two corresponding performance metrics (respectively, MAE and accuracy).- Fitting the model to data- Evaluation the model- Making predictions<jupyter_code>import numpy as np

num_samples = 1280

# initializing inputs 
title_data = np.random.randint(0, 2, size=(num_samples, vocabulary_size))
text_body_data = np.random.randint(0, 2, size=(num_samples, vocabulary_size))
tags_data = np.random.randint(0, 2, size=(num_samples, num_tags))

#initializing outputs
priority_data = np.random.random(size=(num_samples, 1))
department_data = np.random.randint(0, 2, size=(num_samples, num_departments))

model.compile(optimizer="rmsprop",
              loss=["mean_squared_error", "categorical_crossentropy"],
              metrics=[["mean_absolute_error"], ["accuracy"]])
model.fit([title_data, text_body_data, tags_data],
          [priority_data, department_data],
          epochs=1)
model.evaluate([title_data, text_body_data, tags_data],
               [priority_data, department_data])
priority_preds, department_preds = model.predict([title_data, text_body_data, tags_data])<jupyter_output>40/40 [==============================] - 2s 19ms/step - loss: 30.2118 - priority_loss: 0.3407 - department_loss: 29.8711 - priority_mean_absolute_error: 0.5072 - department_accuracy: 0.2289
40/40 [==============================] - 0s 7ms/step - loss: 31.4978 - priority_loss: 0.3446 - department_loss: 31.1532 - priority_mean_absolute_error: 0.5108 - department_accuracy: 0.1234<jupyter_text>**Training a model by providing dicts of input & target arrays**If you have many inputs and outputs, it might be inconvenient to rely on input and output order as in the previous implementation. In that case, the names given to input objects, and to output layers could be leveraged while the data may be passed using dictionaries. This is shown below.<jupyter_code>model.compile(optimizer="rmsprop",
              loss={"priority": "mean_squared_error", "department": "categorical_crossentropy"},
              metrics={"priority": ["mean_absolute_error"], "department": ["accuracy"]})
model.fit({"title": title_data, "text_body": text_body_data, "tags": tags_data},
          {"priority": priority_data, "department": department_data},
          epochs=1)
model.evaluate({"title": title_data, "text_body": text_body_data, "tags": tags_data},
               {"priority": priority_data, "department": department_data})
priority_preds, department_preds = model.predict(
    {"title": title_data, "text_body": text_body_data, "tags": tags_data})<jupyter_output>40/40 [==============================] - 2s 27ms/step - loss: 39.0404 - priority_loss: 0.3446 - department_loss: 38.6957 - priority_mean_absolute_error: 0.5108 - department_accuracy: 0.2586
40/40 [==============================] - 0s 8ms/step - loss: 44.6258 - priority_loss: 0.3446 - department_loss: 44.2811 - priority_mean_absolute_error: 0.5108 - department_accuracy: 0.2688<jupyter_text>The power of the Functional API: Access to layer connectivityThe Functional API creates a graph data structure, which is useful in the following ways:- Visualization of the model (*topology* of the model)- Feature extraction- Nodes from previous graph can be re-used as part of a new model.The plot_model() utility is used to create the graph of "ticket_classifier model" below.  Plotting Layer Connectivity<jupyter_code>keras.utils.plot_model(model, "ticket_classifier.png")<jupyter_output><empty_output><jupyter_text>It is helpful to in debugging to add input and output shapes of layers to the model graph. This is done by setting show_shapes=True. See below.<jupyter_code>keras.utils.plot_model(model, "ticket_classifier_with_shape_info.png", show_shapes=True)<jupyter_output><empty_output><jupyter_text>Feature Extraction With a Fuctuional APIThe layer in the model graph can be inspected (and reused, as we will see later).- Model property model.layers provides a list of layers making up the model.- For each layer, the layer's input and output can be querried, respectively, by layer.input and layer.output. **Retrieving the inputs or outputs of a layer in a Functional model**Below, all layers of the "ticket classifier model" are displayed. In addition, **input** and **output** for the concatenation layer (i.e. the fourth layer or layers[3]) are inspected.<jupyter_code># listing model layers
model.layers
# inspecting input for the concatenation layer (4rth layer i.e. layers[3])
model.layers[3].input
# inspecting output for the concatenation layer
model.layers[3].output<jupyter_output><empty_output><jupyter_text>**Creating a new model by reusing intermediate layer outputs**Suppose the "ticket classifier model" is to be updated by adding a third output, "difficulty", to reflect the time taken to resolve the ticket ('quick', 'medium', 'difficult'). Only the classification layer (output layer) in the previous model needs to be rebuilt. - So the representation of features created at layer[4] -- that is just before the classification layer -- is still valid becuase nothing has changed until that point. - Hence the output from layer[4] can be re-used and fed as features into a new classification-layer that would be built when we respecify the model to have three rather than two outputs.<jupyter_code># reusing layer[4] from previous model
# i.e. declaring representation(output) at layer[4] as new features
features = model.layers[4].output

# passing those features into the new classification layer 
# that produces the output 'difficulty'
difficulty = layers.Dense(3, activation="softmax", name="difficulty")(features)

#re-defining the model with new outputs: 'priority', 'department', 'difficulty'
new_model = keras.Model(
    inputs=[title, text_body, tags],
    outputs=[priority, department, difficulty])
keras.utils.plot_model(new_model, "updated_ticket_classifier.png", show_shapes=True)<jupyter_output><empty_output><jupyter_text>Subclassing the Model class  Rewriting our previous example as a subclassed model **A simple subclassed model**<jupyter_code>class CustomerTicketModel(keras.Model):

    def __init__(self, num_departments):
        super().__init__()
        self.concat_layer = layers.Concatenate()
        self.mixing_layer = layers.Dense(64, activation="relu")
        self.priority_scorer = layers.Dense(1, activation="sigmoid")
        self.department_classifier = layers.Dense(
            num_departments, activation="softmax")

    def call(self, inputs):
        title = inputs["title"]
        text_body = inputs["text_body"]
        tags = inputs["tags"]

        features = self.concat_layer([title, text_body, tags])
        features = self.mixing_layer(features)
        priority = self.priority_scorer(features)
        department = self.department_classifier(features)
        return priority, department
model = CustomerTicketModel(num_departments=4)

priority, department = model(
    {"title": title_data, "text_body": text_body_data, "tags": tags_data})
model.compile(optimizer="rmsprop",
              loss=["mean_squared_error", "categorical_crossentropy"],
              metrics=[["mean_absolute_error"], ["accuracy"]])
model.fit({"title": title_data,
           "text_body": text_body_data,
           "tags": tags_data},
          [priority_data, department_data],
          epochs=1)
model.evaluate({"title": title_data,
                "text_body": text_body_data,
                "tags": tags_data},
               [priority_data, department_data])
priority_preds, department_preds = model.predict({"title": title_data,
                                                  "text_body": text_body_data,
                                                  "tags": tags_data})<jupyter_output><empty_output><jupyter_text>Beware: What subclassed models don't support  Mixing and matching different components **Creating a Functional model that includes a subclassed model**<jupyter_code>class Classifier(keras.Model):

    def __init__(self, num_classes=2):
        super().__init__()
        if num_classes == 2:
            num_units = 1
            activation = "sigmoid"
        else:
            num_units = num_classes
            activation = "softmax"
        self.dense = layers.Dense(num_units, activation=activation)

    def call(self, inputs):
        return self.dense(inputs)

inputs = keras.Input(shape=(3,))
features = layers.Dense(64, activation="relu")(inputs)
outputs = Classifier(num_classes=10)(features)
model = keras.Model(inputs=inputs, outputs=outputs)<jupyter_output><empty_output><jupyter_text>**Creating a subclassed model that includes a Functional model**<jupyter_code>inputs = keras.Input(shape=(64,))
outputs = layers.Dense(1, activation="sigmoid")(inputs)
binary_classifier = keras.Model(inputs=inputs, outputs=outputs)

class MyModel(keras.Model):

    def __init__(self, num_classes=2):
        super().__init__()
        self.dense = layers.Dense(64, activation="relu")
        self.classifier = binary_classifier

    def call(self, inputs):
        features = self.dense(inputs)
        return self.classifier(features)

model = MyModel()<jupyter_output><empty_output><jupyter_text>Remember: Use the right tool for the job  Using built-in training and evaluation loops **The standard workflow: `compile()`, `fit()`, `evaluate()`, `predict()`**<jupyter_code>from tensorflow.keras.datasets import mnist

def get_mnist_model():
    inputs = keras.Input(shape=(28 * 28,))
    features = layers.Dense(512, activation="relu")(inputs)
    features = layers.Dropout(0.5)(features)
    outputs = layers.Dense(10, activation="softmax")(features)
    model = keras.Model(inputs, outputs)
    return model

(images, labels), (test_images, test_labels) = mnist.load_data()
images = images.reshape((60000, 28 * 28)).astype("float32") / 255
test_images = test_images.reshape((10000, 28 * 28)).astype("float32") / 255
train_images, val_images = images[10000:], images[:10000]
train_labels, val_labels = labels[10000:], labels[:10000]

model = get_mnist_model()
model.compile(optimizer="rmsprop",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(train_images, train_labels,
          epochs=3,
          validation_data=(val_images, val_labels))
test_metrics = model.evaluate(test_images, test_labels)
predictions = model.predict(test_images)<jupyter_output><empty_output><jupyter_text>Writing your own metrics **Implementing a custom metric by subclassing the `Metric` class**<jupyter_code>import tensorflow as tf

class RootMeanSquaredError(keras.metrics.Metric):

    def __init__(self, name="rmse", **kwargs):
        super().__init__(name=name, **kwargs)
        self.mse_sum = self.add_weight(name="mse_sum", initializer="zeros")
        self.total_samples = self.add_weight(
            name="total_samples", initializer="zeros", dtype="int32")

    def update_state(self, y_true, y_pred, sample_weight=None):
        y_true = tf.one_hot(y_true, depth=tf.shape(y_pred)[1])
        mse = tf.reduce_sum(tf.square(y_true - y_pred))
        self.mse_sum.assign_add(mse)
        num_samples = tf.shape(y_pred)[0]
        self.total_samples.assign_add(num_samples)

    def result(self):
        return tf.sqrt(self.mse_sum / tf.cast(self.total_samples, tf.float32))

    def reset_state(self):
        self.mse_sum.assign(0.)
        self.total_samples.assign(0)
model = get_mnist_model()
model.compile(optimizer="rmsprop",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy", RootMeanSquaredError()])
model.fit(train_images, train_labels,
          epochs=3,
          validation_data=(val_images, val_labels))
test_metrics = model.evaluate(test_images, test_labels)<jupyter_output><empty_output><jupyter_text>Using callbacks  The EarlyStopping and ModelCheckpoint callbacks **Using the `callbacks` argument in the `fit()` method**<jupyter_code>callbacks_list = [
    keras.callbacks.EarlyStopping(
        monitor="val_accuracy",
        patience=2,
    ),
    keras.callbacks.ModelCheckpoint(
        filepath="checkpoint_path.keras",
        monitor="val_loss",
        save_best_only=True,
    )
]
model = get_mnist_model()
model.compile(optimizer="rmsprop",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(train_images, train_labels,
          epochs=10,
          callbacks=callbacks_list,
          validation_data=(val_images, val_labels))
model = keras.models.load_model("checkpoint_path.keras")<jupyter_output><empty_output><jupyter_text>Writing your own callbacks **Creating a custom callback by subclassing the `Callback` class**<jupyter_code>from matplotlib import pyplot as plt

class LossHistory(keras.callbacks.Callback):
    def on_train_begin(self, logs):
        self.per_batch_losses = []

    def on_batch_end(self, batch, logs):
        self.per_batch_losses.append(logs.get("loss"))

    def on_epoch_end(self, epoch, logs):
        plt.clf()
        plt.plot(range(len(self.per_batch_losses)), self.per_batch_losses,
                 label="Training loss for each batch")
        plt.xlabel(f"Batch (epoch {epoch})")
        plt.ylabel("Loss")
        plt.legend()
        plt.savefig(f"plot_at_epoch_{epoch}")
        self.per_batch_losses = []
model = get_mnist_model()
model.compile(optimizer="rmsprop",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(train_images, train_labels,
          epochs=10,
          callbacks=[LossHistory()],
          validation_data=(val_images, val_labels))<jupyter_output><empty_output><jupyter_text>Monitoring and visualization with TensorBoard<jupyter_code>model = get_mnist_model()
model.compile(optimizer="rmsprop",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])

tensorboard = keras.callbacks.TensorBoard(
    log_dir="/full_path_to_your_log_dir",
)
model.fit(train_images, train_labels,
          epochs=10,
          validation_data=(val_images, val_labels),
          callbacks=[tensorboard])
%load_ext tensorboard
%tensorboard --logdir /full_path_to_your_log_dir<jupyter_output><empty_output><jupyter_text>Writing your own training and evaluation loops  Training versus inference  Low-level usage of metrics<jupyter_code>metric = keras.metrics.SparseCategoricalAccuracy()
targets = [0, 1, 2]
predictions = [[1, 0, 0], [0, 1, 0], [0, 0, 1]]
metric.update_state(targets, predictions)
current_result = metric.result()
print(f"result: {current_result:.2f}")
values = [0, 1, 2, 3, 4]
mean_tracker = keras.metrics.Mean()
for value in values:
    mean_tracker.update_state(value)
print(f"Mean of values: {mean_tracker.result():.2f}")<jupyter_output><empty_output><jupyter_text>A complete training and evaluation loop **Writing a step-by-step training loop: the training step function**<jupyter_code>model = get_mnist_model()

loss_fn = keras.losses.SparseCategoricalCrossentropy()
optimizer = keras.optimizers.RMSprop()
metrics = [keras.metrics.SparseCategoricalAccuracy()]
loss_tracking_metric = keras.metrics.Mean()

def train_step(inputs, targets):
    with tf.GradientTape() as tape:
        predictions = model(inputs, training=True)
        loss = loss_fn(targets, predictions)
    gradients = tape.gradient(loss, model.trainable_weights)
    optimizer.apply_gradients(zip(gradients, model.trainable_weights))

    logs = {}
    for metric in metrics:
        metric.update_state(targets, predictions)
        logs[metric.name] = metric.result()

    loss_tracking_metric.update_state(loss)
    logs["loss"] = loss_tracking_metric.result()
    return logs<jupyter_output><empty_output><jupyter_text>**Writing a step-by-step training loop: resetting the metrics**<jupyter_code>def reset_metrics():
    for metric in metrics:
        metric.reset_state()
    loss_tracking_metric.reset_state()<jupyter_output><empty_output><jupyter_text>**Writing a step-by-step training loop: the loop itself**<jupyter_code>training_dataset = tf.data.Dataset.from_tensor_slices((train_images, train_labels))
training_dataset = training_dataset.batch(32)
epochs = 3
for epoch in range(epochs):
    reset_metrics()
    for inputs_batch, targets_batch in training_dataset:
        logs = train_step(inputs_batch, targets_batch)
    print(f"Results at the end of epoch {epoch}")
    for key, value in logs.items():
        print(f"...{key}: {value:.4f}")<jupyter_output><empty_output><jupyter_text>**Writing a step-by-step evaluation loop**<jupyter_code>def test_step(inputs, targets):
    predictions = model(inputs, training=False)
    loss = loss_fn(targets, predictions)

    logs = {}
    for metric in metrics:
        metric.update_state(targets, predictions)
        logs["val_" + metric.name] = metric.result()

    loss_tracking_metric.update_state(loss)
    logs["val_loss"] = loss_tracking_metric.result()
    return logs

val_dataset = tf.data.Dataset.from_tensor_slices((val_images, val_labels))
val_dataset = val_dataset.batch(32)
reset_metrics()
for inputs_batch, targets_batch in val_dataset:
    logs = test_step(inputs_batch, targets_batch)
print("Evaluation results:")
for key, value in logs.items():
    print(f"...{key}: {value:.4f}")<jupyter_output><empty_output><jupyter_text>Make it fast with tf.function **Adding a `tf.function` decorator to our evaluation-step function**<jupyter_code>@tf.function
def test_step(inputs, targets):
    predictions = model(inputs, training=False)
    loss = loss_fn(targets, predictions)

    logs = {}
    for metric in metrics:
        metric.update_state(targets, predictions)
        logs["val_" + metric.name] = metric.result()

    loss_tracking_metric.update_state(loss)
    logs["val_loss"] = loss_tracking_metric.result()
    return logs

val_dataset = tf.data.Dataset.from_tensor_slices((val_images, val_labels))
val_dataset = val_dataset.batch(32)
reset_metrics()
for inputs_batch, targets_batch in val_dataset:
    logs = test_step(inputs_batch, targets_batch)
print("Evaluation results:")
for key, value in logs.items():
    print(f"...{key}: {value:.4f}")<jupyter_output><empty_output><jupyter_text>Leveraging fit() with a custom training loop **Implementing a custom training step to use with `fit()`**<jupyter_code>loss_fn = keras.losses.SparseCategoricalCrossentropy()
loss_tracker = keras.metrics.Mean(name="loss")

class CustomModel(keras.Model):
    def train_step(self, data):
        inputs, targets = data
        with tf.GradientTape() as tape:
            predictions = self(inputs, training=True)
            loss = loss_fn(targets, predictions)
        gradients = tape.gradient(loss, model.trainable_weights)
        optimizer.apply_gradients(zip(gradients, model.trainable_weights))

        loss_tracker.update_state(loss)
        return {"loss": loss_tracker.result()}

    @property
    def metrics(self):
        return [loss_tracker]
inputs = keras.Input(shape=(28 * 28,))
features = layers.Dense(512, activation="relu")(inputs)
features = layers.Dropout(0.5)(features)
outputs = layers.Dense(10, activation="softmax")(features)
model = CustomModel(inputs, outputs)

model.compile(optimizer=keras.optimizers.RMSprop())
model.fit(train_images, train_labels, epochs=3)
class CustomModel(keras.Model):
    def train_step(self, data):
        inputs, targets = data
        with tf.GradientTape() as tape:
            predictions = self(inputs, training=True)
            loss = self.compiled_loss(targets, predictions)
        gradients = tape.gradient(loss, model.trainable_weights)
        optimizer.apply_gradients(zip(gradients, model.trainable_weights))
        self.compiled_metrics.update_state(targets, predictions)
        return {m.name: m.result() for m in self.metrics}
inputs = keras.Input(shape=(28 * 28,))
features = layers.Dense(512, activation="relu")(inputs)
features = layers.Dropout(0.5)(features)
outputs = layers.Dense(10, activation="softmax")(features)
model = CustomModel(inputs, outputs)

model.compile(optimizer=keras.optimizers.RMSprop(),
              loss=keras.losses.SparseCategoricalCrossentropy(),
              metrics=[keras.metrics.SparseCategoricalAccuracy()])
model.fit(train_images, train_labels, epochs=3)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1175456739055498, 'fraction_numerical': 0.014509103443953496, 'mean_word_length': 4.215231246935774, 'pattern_counts': {'":': 30, '<': 115, '<?xml version=': 0, '>': 117, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 22, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '30', 'dir': 'jupyter-structured-clean-dedup', 'id': '557851', 'license': 'MIT', 'n_tokens_mistral': 9338, 'n_tokens_neox': 8789, 'n_words': 2619, 'path': 'chapter07_working-with-keras.ipynb', 'repo_name': 'kriaz100/deep-learning-with-python-notebooks', 'size': '31911'}