cell class for simplernn this class processes one step within the whole time sequence input whereas keras layer simplernn processes the whole sequence args units positive integer dimensionality of the output space activation activation function to use default hyperbolic tangent tanh if you pass none no activation is applied ie linear activation ax x usebias boolean default true whether the layer should use a bias vector kernelinitializer initializer for the kernel weights matrix used for the linear transformation of the inputs default glorotuniform recurrentinitializer initializer for the recurrentkernel weights matrix used for the linear transformation of the recurrent state default orthogonal biasinitializer initializer for the bias vector default zeros kernelregularizer regularizer function applied to the kernel weights matrix default none recurrentregularizer regularizer function applied to the recurrentkernel weights matrix default none biasregularizer regularizer function applied to the bias vector default none kernelconstraint constraint function applied to the kernel weights matrix default none recurrentconstraint constraint function applied to the recurrentkernel weights matrix default none biasconstraint constraint function applied to the bias vector default none dropout float between 0 and 1 fraction of the units to drop for the linear transformation of the inputs default 0 recurrentdropout float between 0 and 1 fraction of the units to drop for the linear transformation of the recurrent state default 0 seed random seed for dropout call arguments sequence a 2d tensor with shape batch features states a 2d tensor with shape batch units which is the state from the previous time step training python boolean indicating whether the layer should behave in training mode or in inference mode only relevant when dropout or recurrentdropout is used example python inputs np random random32 10 8 astypenp float32 rnn keras layers rnnkeras layers simplernncell4 output rnninputs the output has shape 32 4 rnn keras layers rnn keras layers simplernncell4 returnsequencestrue returnstatetrue wholesequenceoutput has shape 32 10 4 finalstate has shape 32 4 wholesequenceoutput finalstate rnninputs fullyconnected rnn where the output is to be fed back as the new input args units positive integer dimensionality of the output space activation activation function to use default hyperbolic tangent tanh if you pass none no activation is applied ie linear activation ax x usebias boolean default true whether the layer uses a bias vector kernelinitializer initializer for the kernel weights matrix used for the linear transformation of the inputs default glorotuniform recurrentinitializer initializer for the recurrentkernel weights matrix used for the linear transformation of the recurrent state default orthogonal biasinitializer initializer for the bias vector default zeros kernelregularizer regularizer function applied to the kernel weights matrix default none recurrentregularizer regularizer function applied to the recurrentkernel weights matrix default none biasregularizer regularizer function applied to the bias vector default none activityregularizer regularizer function applied to the output of the layer its activation default none kernelconstraint constraint function applied to the kernel weights matrix default none recurrentconstraint constraint function applied to the recurrentkernel weights matrix default none biasconstraint constraint function applied to the bias vector default none dropout float between 0 and 1 fraction of the units to drop for the linear transformation of the inputs default 0 recurrentdropout float between 0 and 1 fraction of the units to drop for the linear transformation of the recurrent state default 0 returnsequences boolean whether to return the last output in the output sequence or the full sequence default false returnstate boolean whether to return the last state in addition to the output default false gobackwards boolean default false if true process the input sequence backwards and return the reversed sequence stateful boolean default false if true the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch unroll boolean default false if true the network will be unrolled else a symbolic loop will be used unrolling can speedup a rnn although it tends to be more memoryintensive unrolling is only suitable for short sequences call arguments sequence a 3d tensor with shape batch timesteps feature mask binary tensor of shape batch timesteps indicating whether a given timestep should be masked an individual true entry indicates that the corresponding timestep should be utilized while a false entry indicates that the corresponding timestep should be ignored training python boolean indicating whether the layer should behave in training mode or in inference mode this argument is passed to the cell when calling it this is only relevant if dropout or recurrentdropout is used initialstate list of initial state tensors to be passed to the first call of the cell example python inputs np random random32 10 8 simplernn keras layers simplernn4 output simplernninputs the output has shape 32 4 simplernn keras layers simplernn 4 returnsequencestrue returnstatetrue wholesequenceoutput has shape 32 10 4 finalstate has shape 32 4 wholesequenceoutput finalstate simplernninputs cell class for simplernn this class processes one step within the whole time sequence input whereas keras layer simplernn processes the whole sequence args units positive integer dimensionality of the output space activation activation function to use default hyperbolic tangent tanh if you pass none no activation is applied ie linear activation a x x use_bias boolean default true whether the layer should use a bias vector kernel_initializer initializer for the kernel weights matrix used for the linear transformation of the inputs default glorot_uniform recurrent_initializer initializer for the recurrent_kernel weights matrix used for the linear transformation of the recurrent state default orthogonal bias_initializer initializer for the bias vector default zeros kernel_regularizer regularizer function applied to the kernel weights matrix default none recurrent_regularizer regularizer function applied to the recurrent_kernel weights matrix default none bias_regularizer regularizer function applied to the bias vector default none kernel_constraint constraint function applied to the kernel weights matrix default none recurrent_constraint constraint function applied to the recurrent_kernel weights matrix default none bias_constraint constraint function applied to the bias vector default none dropout float between 0 and 1 fraction of the units to drop for the linear transformation of the inputs default 0 recurrent_dropout float between 0 and 1 fraction of the units to drop for the linear transformation of the recurrent state default 0 seed random seed for dropout call arguments sequence a 2d tensor with shape batch features states a 2d tensor with shape batch units which is the state from the previous time step training python boolean indicating whether the layer should behave in training mode or in inference mode only relevant when dropout or recurrent_dropout is used example python inputs np random random 32 10 8 astype np float32 rnn keras layers rnn keras layers simplernncell 4 output rnn inputs the output has shape 32 4 rnn keras layers rnn keras layers simplernncell 4 return_sequences true return_state true whole_sequence_output has shape 32 10 4 final_state has shape 32 4 whole_sequence_output final_state rnn inputs fully connected rnn where the output is to be fed back as the new input args units positive integer dimensionality of the output space activation activation function to use default hyperbolic tangent tanh if you pass none no activation is applied ie linear activation a x x use_bias boolean default true whether the layer uses a bias vector kernel_initializer initializer for the kernel weights matrix used for the linear transformation of the inputs default glorot_uniform recurrent_initializer initializer for the recurrent_kernel weights matrix used for the linear transformation of the recurrent state default orthogonal bias_initializer initializer for the bias vector default zeros kernel_regularizer regularizer function applied to the kernel weights matrix default none recurrent_regularizer regularizer function applied to the recurrent_kernel weights matrix default none bias_regularizer regularizer function applied to the bias vector default none activity_regularizer regularizer function applied to the output of the layer its activation default none kernel_constraint constraint function applied to the kernel weights matrix default none recurrent_constraint constraint function applied to the recurrent_kernel weights matrix default none bias_constraint constraint function applied to the bias vector default none dropout float between 0 and 1 fraction of the units to drop for the linear transformation of the inputs default 0 recurrent_dropout float between 0 and 1 fraction of the units to drop for the linear transformation of the recurrent state default 0 return_sequences boolean whether to return the last output in the output sequence or the full sequence default false return_state boolean whether to return the last state in addition to the output default false go_backwards boolean default false if true process the input sequence backwards and return the reversed sequence stateful boolean default false if true the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch unroll boolean default false if true the network will be unrolled else a symbolic loop will be used unrolling can speed up a rnn although it tends to be more memory intensive unrolling is only suitable for short sequences call arguments sequence a 3d tensor with shape batch timesteps feature mask binary tensor of shape batch timesteps indicating whether a given timestep should be masked an individual true entry indicates that the corresponding timestep should be utilized while a false entry indicates that the corresponding timestep should be ignored training python boolean indicating whether the layer should behave in training mode or in inference mode this argument is passed to the cell when calling it this is only relevant if dropout or recurrent_dropout is used initial_state list of initial state tensors to be passed to the first call of the cell example python inputs np random random 32 10 8 simple_rnn keras layers simplernn 4 output simple_rnn inputs the output has shape 32 4 simple_rnn keras layers simplernn 4 return_sequences true return_state true whole_sequence_output has shape 32 10 4 final_state has shape 32 4 whole_sequence_output final_state simple_rnn inputs
from keras import activations
from keras import backend
from keras import constraints
from keras import initializers
from keras import ops
from keras import regularizers
from keras.api_export import keras_export
from keras.layers.input_spec import InputSpec
from keras.layers.layer import Layer
from keras.layers.rnn.dropout_rnn_cell import DropoutRNNCell
from keras.layers.rnn.rnn import RNN


@keras_export("keras.layers.SimpleRNNCell")
class SimpleRNNCell(Layer, DropoutRNNCell):
    

    def __init__(
        self,
        units,
        activation="tanh",
        use_bias=True,
        kernel_initializer="glorot_uniform",
        recurrent_initializer="orthogonal",
        bias_initializer="zeros",
        kernel_regularizer=None,
        recurrent_regularizer=None,
        bias_regularizer=None,
        kernel_constraint=None,
        recurrent_constraint=None,
        bias_constraint=None,
        dropout=0.0,
        recurrent_dropout=0.0,
        seed=None,
        **kwargs,
    ):
        if units <= 0:
            raise ValueError(
                "Received an invalid value for argument `units`, "
                f"expected a positive integer, got {units}."
            )
        super().__init__(**kwargs)
        self.seed = seed
        self.seed_generator = backend.random.SeedGenerator(seed)

        self.units = units
        self.activation = activations.get(activation)
        self.use_bias = use_bias

        self.kernel_initializer = initializers.get(kernel_initializer)
        self.recurrent_initializer = initializers.get(recurrent_initializer)
        self.bias_initializer = initializers.get(bias_initializer)

        self.kernel_regularizer = regularizers.get(kernel_regularizer)
        self.recurrent_regularizer = regularizers.get(recurrent_regularizer)
        self.bias_regularizer = regularizers.get(bias_regularizer)

        self.kernel_constraint = constraints.get(kernel_constraint)
        self.recurrent_constraint = constraints.get(recurrent_constraint)
        self.bias_constraint = constraints.get(bias_constraint)

        self.dropout = min(1.0, max(0.0, dropout))
        self.recurrent_dropout = min(1.0, max(0.0, recurrent_dropout))
        self.state_size = self.units
        self.output_size = self.units

    def build(self, input_shape):
        self.kernel = self.add_weight(
            shape=(input_shape[-1], self.units),
            name="kernel",
            initializer=self.kernel_initializer,
            regularizer=self.kernel_regularizer,
            constraint=self.kernel_constraint,
        )
        self.recurrent_kernel = self.add_weight(
            shape=(self.units, self.units),
            name="recurrent_kernel",
            initializer=self.recurrent_initializer,
            regularizer=self.recurrent_regularizer,
            constraint=self.recurrent_constraint,
        )
        if self.use_bias:
            self.bias = self.add_weight(
                shape=(self.units,),
                name="bias",
                initializer=self.bias_initializer,
                regularizer=self.bias_regularizer,
                constraint=self.bias_constraint,
            )
        else:
            self.bias = None
        self.built = True

    def call(self, sequence, states, training=False):
        prev_output = states[0] if isinstance(states, (list, tuple)) else states
        dp_mask = self.get_dropout_mask(sequence)
        rec_dp_mask = self.get_recurrent_dropout_mask(prev_output)

        if training and dp_mask is not None:
            sequence *= dp_mask
        h = ops.matmul(sequence, self.kernel)
        if self.bias is not None:
            h += self.bias

        if training and rec_dp_mask is not None:
            prev_output *= rec_dp_mask
        output = h + ops.matmul(prev_output, self.recurrent_kernel)
        if self.activation is not None:
            output = self.activation(output)

        new_state = [output] if isinstance(states, (list, tuple)) else output
        return output, new_state

    def get_initial_state(self, batch_size=None):
        return [
            ops.zeros((batch_size, self.state_size), dtype=self.compute_dtype)
        ]

    def get_config(self):
        config = {
            "units": self.units,
            "activation": activations.serialize(self.activation),
            "use_bias": self.use_bias,
            "kernel_initializer": initializers.serialize(
                self.kernel_initializer
            ),
            "recurrent_initializer": initializers.serialize(
                self.recurrent_initializer
            ),
            "bias_initializer": initializers.serialize(self.bias_initializer),
            "kernel_regularizer": regularizers.serialize(
                self.kernel_regularizer
            ),
            "recurrent_regularizer": regularizers.serialize(
                self.recurrent_regularizer
            ),
            "bias_regularizer": regularizers.serialize(self.bias_regularizer),
            "kernel_constraint": constraints.serialize(self.kernel_constraint),
            "recurrent_constraint": constraints.serialize(
                self.recurrent_constraint
            ),
            "bias_constraint": constraints.serialize(self.bias_constraint),
            "dropout": self.dropout,
            "recurrent_dropout": self.recurrent_dropout,
            "seed": self.seed,
        }
        base_config = super().get_config()
        return {**base_config, **config}


@keras_export("keras.layers.SimpleRNN")
class SimpleRNN(RNN):
    

    def __init__(
        self,
        units,
        activation="tanh",
        use_bias=True,
        kernel_initializer="glorot_uniform",
        recurrent_initializer="orthogonal",
        bias_initializer="zeros",
        kernel_regularizer=None,
        recurrent_regularizer=None,
        bias_regularizer=None,
        activity_regularizer=None,
        kernel_constraint=None,
        recurrent_constraint=None,
        bias_constraint=None,
        dropout=0.0,
        recurrent_dropout=0.0,
        return_sequences=False,
        return_state=False,
        go_backwards=False,
        stateful=False,
        unroll=False,
        seed=None,
        **kwargs,
    ):
        cell = SimpleRNNCell(
            units,
            activation=activation,
            use_bias=use_bias,
            kernel_initializer=kernel_initializer,
            recurrent_initializer=recurrent_initializer,
            bias_initializer=bias_initializer,
            kernel_regularizer=kernel_regularizer,
            recurrent_regularizer=recurrent_regularizer,
            bias_regularizer=bias_regularizer,
            kernel_constraint=kernel_constraint,
            recurrent_constraint=recurrent_constraint,
            bias_constraint=bias_constraint,
            dropout=dropout,
            recurrent_dropout=recurrent_dropout,
            seed=seed,
            dtype=kwargs.get("dtype", None),
            trainable=kwargs.get("trainable", True),
            name="simple_rnn_cell",
        )
        super().__init__(
            cell,
            return_sequences=return_sequences,
            return_state=return_state,
            go_backwards=go_backwards,
            stateful=stateful,
            unroll=unroll,
            **kwargs,
        )
        self.input_spec = [InputSpec(ndim=3)]

    def call(self, sequences, initial_state=None, mask=None, training=False):
        return super().call(
            sequences, mask=mask, training=training, initial_state=initial_state
        )

    @property
    def units(self):
        return self.cell.units

    @property
    def activation(self):
        return self.cell.activation

    @property
    def use_bias(self):
        return self.cell.use_bias

    @property
    def kernel_initializer(self):
        return self.cell.kernel_initializer

    @property
    def recurrent_initializer(self):
        return self.cell.recurrent_initializer

    @property
    def bias_initializer(self):
        return self.cell.bias_initializer

    @property
    def kernel_regularizer(self):
        return self.cell.kernel_regularizer

    @property
    def recurrent_regularizer(self):
        return self.cell.recurrent_regularizer

    @property
    def bias_regularizer(self):
        return self.cell.bias_regularizer

    @property
    def kernel_constraint(self):
        return self.cell.kernel_constraint

    @property
    def recurrent_constraint(self):
        return self.cell.recurrent_constraint

    @property
    def bias_constraint(self):
        return self.cell.bias_constraint

    @property
    def dropout(self):
        return self.cell.dropout

    @property
    def recurrent_dropout(self):
        return self.cell.recurrent_dropout

    def get_config(self):
        config = {
            "units": self.units,
            "activation": activations.serialize(self.activation),
            "use_bias": self.use_bias,
            "kernel_initializer": initializers.serialize(
                self.kernel_initializer
            ),
            "recurrent_initializer": initializers.serialize(
                self.recurrent_initializer
            ),
            "bias_initializer": initializers.serialize(self.bias_initializer),
            "kernel_regularizer": regularizers.serialize(
                self.kernel_regularizer
            ),
            "recurrent_regularizer": regularizers.serialize(
                self.recurrent_regularizer
            ),
            "bias_regularizer": regularizers.serialize(self.bias_regularizer),
            "activity_regularizer": regularizers.serialize(
                self.activity_regularizer
            ),
            "kernel_constraint": constraints.serialize(self.kernel_constraint),
            "recurrent_constraint": constraints.serialize(
                self.recurrent_constraint
            ),
            "bias_constraint": constraints.serialize(self.bias_constraint),
            "dropout": self.dropout,
            "recurrent_dropout": self.recurrent_dropout,
        }
        base_config = super().get_config()
        del base_config["cell"]
        return {**base_config, **config}

    @classmethod
    def from_config(cls, config):
        return cls(**config)