title the sequential model fchollethttps twitter comfchollet date created 20200412 last modified 20230625 description complete guide to the sequential model accelerator gpu setup when to use a sequential model a sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor schematically the following sequential model define sequential model with 3 layers call model on a test input is equivalent to this function create 3 layers call layers on a test input a sequential model is not appropriate when your model has multiple inputs or multiple outputs any of your layers has multiple inputs or multiple outputs you need to do layer sharing you want nonlinear topology e g a residual connection a multibranch model creating a sequential model you can create a sequential model by passing a list of layers to the sequential constructor its layers are accessible via the layers attribute you can also create a sequential model incrementally via the add method note that there s also a corresponding pop method to remove layers a sequential model behaves very much like a list of layers also note that the sequential constructor accepts a name argument just like any layer or model in keras this is useful to annotate tensorboard graphs with semantically meaningful names specifying the input shape in advance generally all layers in keras need to know the shape of their inputs in order to be able to create their weights so when you create a layer like this initially it has no weights it creates its weights the first time it is called on an input since the shape of the weights depends on the shape of the inputs call layer on a test input naturally this also applies to sequential models when you instantiate a sequential model without an input shape it isn t built it has no weights and calling model weights results in an error stating just this the weights are created when the model first sees some input data at this point you can t do this model weights you also can t do this model summary call the model on a test input once a model is built you can call its summary method to display its contents however it can be very useful when building a sequential model incrementally to be able to display the summary of the model so far including the current output shape in this case you should start your model by passing an input object to your model so that it knows its input shape from the start note that the input object is not displayed as part of model layers since it isn t a layer models built with a predefined input shape like this always have weights even before seeing any data and always have a defined output shape in general it s a recommended best practice to always specify the input shape of a sequential model in advance if you know what it is a common debugging workflow add summary when building a new sequential architecture it s useful to incrementally stack layers with add and frequently print model summaries for instance this enables you to monitor how a stack of conv2d and maxpooling2d layers is downsampling image feature maps can you guess what the current output shape is at this point probably not let s just print it the answer was 40 40 32 so we can keep downsampling and now now that we have 4x4 feature maps time to apply global max pooling finally we add a classification layer very practical right what to do once you have a model once your model architecture is ready you will want to train your model evaluate it and run inference see our guide to training evaluation with the builtin loops guidestrainingwithbuiltinmethods save your model to disk and restore it see our guide to serialization savingguidesserializationandsaving speed up model training by leveraging multiple gpus see our guide to multigpu and distributed traininghttps keras ioguidesdistributedtraining feature extraction with a sequential model once a sequential model has been built it behaves like a functional api modelguidesfunctionalapi this means that every layer has an input and output attribute these attributes can be used to do neat things like quickly creating a model that extracts the outputs of all intermediate layers in a sequential model call feature extractor on test input here s a similar example that only extract features from one layer call feature extractor on test input transfer learning with a sequential model transfer learning consists of freezing the bottom layers in a model and only training the top layers if you aren t familiar with it make sure to read our guide to transfer learningguidestransferlearning here are two common transfer learning blueprint involving sequential models first let s say that you have a sequential model and you want to freeze all layers except the last one in this case you would simply iterate over model layers and set layer trainable false on each layer except the last one like this python model keras sequential keras inputshape784 layers dense32 activation relu layers dense32 activation relu layers dense32 activation relu layers dense10 presumably you would want to first load pretrained weights model loadweights freeze all layers except the last one for layer in model layers 1 layer trainable false recompile and train this will only update the weights of the last layer model compile model fit another common blueprint is to use a sequential model to stack a pretrained model and some freshly initialized classification layers like this python load a convolutional base with pretrained weights basemodel keras applications xception weights imagenet includetopfalse pooling avg freeze the base model basemodel trainable false use a sequential model to add a trainable classifier on top model keras sequential basemodel layers dense1000 compile train model compile model fit if you do transfer learning you will probably find yourself frequently using these two patterns that s about all you need to know about sequential models to find out more about building models in keras see guide to the functional apiguidesfunctionalapi guide to making new layers models via subclassing guidesmakingnewlayersandmodelsviasubclassing setup when to use a sequential model a sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor schematically the following sequential model define sequential model with 3 layers call model on a test input is equivalent to this function create 3 layers call layers on a test input a sequential model is not appropriate when your model has multiple inputs or multiple outputs any of your layers has multiple inputs or multiple outputs you need to do layer sharing you want non linear topology e g a residual connection a multi branch model creating a sequential model you can create a sequential model by passing a list of layers to the sequential constructor its layers are accessible via the layers attribute you can also create a sequential model incrementally via the add method note that there s also a corresponding pop method to remove layers a sequential model behaves very much like a list of layers 2 also note that the sequential constructor accepts a name argument just like any layer or model in keras this is useful to annotate tensorboard graphs with semantically meaningful names specifying the input shape in advance generally all layers in keras need to know the shape of their inputs in order to be able to create their weights so when you create a layer like this initially it has no weights empty it creates its weights the first time it is called on an input since the shape of the weights depends on the shape of the inputs call layer on a test input now it has weights of shape 4 3 and 3 naturally this also applies to sequential models when you instantiate a sequential model without an input shape it isn t built it has no weights and calling model weights results in an error stating just this the weights are created when the model first sees some input data no weights at this stage at this point you can t do this model weights you also can t do this model summary call the model on a test input 6 once a model is built you can call its summary method to display its contents however it can be very useful when building a sequential model incrementally to be able to display the summary of the model so far including the current output shape in this case you should start your model by passing an input object to your model so that it knows its input shape from the start note that the input object is not displayed as part of model layers since it isn t a layer models built with a predefined input shape like this always have weights even before seeing any data and always have a defined output shape in general it s a recommended best practice to always specify the input shape of a sequential model in advance if you know what it is a common debugging workflow add summary when building a new sequential architecture it s useful to incrementally stack layers with add and frequently print model summaries for instance this enables you to monitor how a stack of conv2d and maxpooling2d layers is downsampling image feature maps 250x250 rgb images can you guess what the current output shape is at this point probably not let s just print it the answer was 40 40 32 so we can keep downsampling and now now that we have 4x4 feature maps time to apply global max pooling finally we add a classification layer very practical right what to do once you have a model once your model architecture is ready you will want to train your model evaluate it and run inference see our guide to training evaluation with the built in loops guides training_with_built_in_methods save your model to disk and restore it see our guide to serialization saving guides serialization_and_saving speed up model training by leveraging multiple gpus see our guide to multi gpu and distributed training https keras io guides distributed_training feature extraction with a sequential model once a sequential model has been built it behaves like a functional api model guides functional_api this means that every layer has an input and output attribute these attributes can be used to do neat things like quickly creating a model that extracts the outputs of all intermediate layers in a sequential model call feature extractor on test input here s a similar example that only extract features from one layer call feature extractor on test input transfer learning with a sequential model transfer learning consists of freezing the bottom layers in a model and only training the top layers if you aren t familiar with it make sure to read our guide to transfer learning guides transfer_learning here are two common transfer learning blueprint involving sequential models first let s say that you have a sequential model and you want to freeze all layers except the last one in this case you would simply iterate over model layers and set layer trainable false on each layer except the last one like this python model keras sequential keras input shape 784 layers dense 32 activation relu layers dense 32 activation relu layers dense 32 activation relu layers dense 10 presumably you would want to first load pre trained weights model load_weights freeze all layers except the last one for layer in model layers 1 layer trainable false recompile and train this will only update the weights of the last layer model compile model fit another common blueprint is to use a sequential model to stack a pre trained model and some freshly initialized classification layers like this python load a convolutional base with pre trained weights base_model keras applications xception weights imagenet include_top false pooling avg freeze the base model base_model trainable false use a sequential model to add a trainable classifier on top model keras sequential base_model layers dense 1000 compile train model compile model fit if you do transfer learning you will probably find yourself frequently using these two patterns that s about all you need to know about sequential models to find out more about building models in keras see guide to the functional api guides functional_api guide to making new layers models via subclassing guides making_new_layers_and_models_via_subclassing
import keras
from keras import layers
from keras import ops




model = keras.Sequential(
    [
        layers.Dense(2, activation="relu", name="layer1"),
        layers.Dense(3, activation="relu", name="layer2"),
        layers.Dense(4, name="layer3"),
    ]
)

x = ops.ones((3, 3))
y = model(x)




layer1 = layers.Dense(2, activation="relu", name="layer1")
layer2 = layers.Dense(3, activation="relu", name="layer2")
layer3 = layers.Dense(4, name="layer3")


x = ops.ones((3, 3))
y = layer3(layer2(layer1(x)))





model = keras.Sequential(
    [
        layers.Dense(2, activation="relu"),
        layers.Dense(3, activation="relu"),
        layers.Dense(4),
    ]
)



model.layers



model = keras.Sequential()
model.add(layers.Dense(2, activation="relu"))
model.add(layers.Dense(3, activation="relu"))
model.add(layers.Dense(4))



model.pop()
print(len(model.layers))  



model = keras.Sequential(name="my_sequential")
model.add(layers.Dense(2, activation="relu", name="layer1"))
model.add(layers.Dense(3, activation="relu", name="layer2"))
model.add(layers.Dense(4, name="layer3"))



layer = layers.Dense(3)
layer.weights  




x = ops.ones((1, 4))
y = layer(x)
layer.weights  



model = keras.Sequential(
    [
        layers.Dense(2, activation="relu"),
        layers.Dense(3, activation="relu"),
        layers.Dense(4),
    ]
)  








x = ops.ones((1, 4))
y = model(x)
print("Number of weights after calling the model:", len(model.weights))  



model.summary()



model = keras.Sequential()
model.add(keras.Input(shape=(4,)))
model.add(layers.Dense(2, activation="relu"))

model.summary()



model.layers





model = keras.Sequential()
model.add(keras.Input(shape=(250, 250, 3)))  
model.add(layers.Conv2D(32, 5, strides=2, activation="relu"))
model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.MaxPooling2D(3))



model.summary()



model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.MaxPooling2D(3))
model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.Conv2D(32, 3, activation="relu"))
model.add(layers.MaxPooling2D(2))


model.summary()


model.add(layers.GlobalMaxPooling2D())


model.add(layers.Dense(10))







initial_model = keras.Sequential(
    [
        keras.Input(shape=(250, 250, 3)),
        layers.Conv2D(32, 5, strides=2, activation="relu"),
        layers.Conv2D(32, 3, activation="relu"),
        layers.Conv2D(32, 3, activation="relu"),
    ]
)
feature_extractor = keras.Model(
    inputs=initial_model.inputs,
    outputs=[layer.output for layer in initial_model.layers],
)


x = ops.ones((1, 250, 250, 3))
features = feature_extractor(x)



initial_model = keras.Sequential(
    [
        keras.Input(shape=(250, 250, 3)),
        layers.Conv2D(32, 5, strides=2, activation="relu"),
        layers.Conv2D(32, 3, activation="relu", name="my_intermediate_layer"),
        layers.Conv2D(32, 3, activation="relu"),
    ]
)
feature_extractor = keras.Model(
    inputs=initial_model.inputs,
    outputs=initial_model.get_layer(name="my_intermediate_layer").output,
)

x = ops.ones((1, 250, 250, 3))
features = feature_extractor(x)