random normal initializer draws samples from a normal distribution for given parameters examples standalone usage initializer randomnormalmean0 0 stddev1 0 values initializershape2 2 usage in a keras layer initializer randomnormalmean0 0 stddev1 0 layer dense3 kernelinitializerinitializer args mean a python scalar or a scalar keras tensor mean of the random values to generate stddev a python scalar or a scalar keras tensor standard deviation of the random values to generate seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator initializer that generates a truncated normal distribution the values generated are similar to values from a randomnormal initializer except that values more than two standard deviations from the mean are discarded and redrawn examples standalone usage initializer truncatednormalmean0 stddev1 values initializershape2 2 usage in a keras layer initializer truncatednormalmean0 stddev1 layer dense3 kernelinitializerinitializer args mean a python scalar or a scalar keras tensor mean of the random values to generate stddev a python scalar or a scalar keras tensor standard deviation of the random values to generate seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator random uniform initializer draws samples from a uniform distribution for given parameters examples standalone usage initializer randomuniformminval0 0 maxval1 0 values initializershape2 2 usage in a keras layer initializer randomuniformminval0 0 maxval1 0 layer dense3 kernelinitializerinitializer args minval a python scalar or a scalar keras tensor lower bound of the range of random values to generate inclusive maxval a python scalar or a scalar keras tensor upper bound of the range of random values to generate exclusive seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator initializer that adapts its scale to the shape of its input tensors with distributiontruncatednormal or untruncatednormal samples are drawn from a truncateduntruncated normal distribution with a mean of zero and a standard deviation after truncation if used stddev sqrtscale n where n is number of input units in the weight tensor if modefanin number of output units if modefanout average of the numbers of input and output units if modefanavg with distributionuniform samples are drawn from a uniform distribution within limit limit where limit sqrt3 scale n examples standalone usage initializer variancescaling scale0 1 mode fanin distribution uniform values initializershape2 2 usage in a keras layer initializer variancescaling scale0 1 mode fanin distribution uniform layer dense3 kernelinitializerinitializer args scale scaling factor positive float mode one of fanin fanout fanavg distribution random distribution to use one of truncatednormal untruncatednormal or uniform seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator the glorot uniform initializer also called xavier uniform initializer draws samples from a uniform distribution within limit limit where limit sqrt6 fanin fanout fanin is the number of input units in the weight tensor and fanout is the number of output units examples standalone usage initializer glorotuniform values initializershape2 2 usage in a keras layer initializer glorotuniform layer dense3 kernelinitializerinitializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference glorot et al 2010http proceedings mlr pressv9glorot10a html the glorot normal initializer also called xavier normal initializer draws samples from a truncated normal distribution centered on 0 with stddev sqrt2 fanin fanout where fanin is the number of input units in the weight tensor and fanout is the number of output units in the weight tensor examples standalone usage initializer glorotnormal values initializershape2 2 usage in a keras layer initializer glorotnormal layer dense3 kernelinitializerinitializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference glorot et al 2010http proceedings mlr pressv9glorot10a html lecun normal initializer initializers allow you to prespecify an initialization strategy encoded in the initializer object without knowing the shape and dtype of the variable being initialized draws samples from a truncated normal distribution centered on 0 with stddev sqrt1 fanin where fanin is the number of input units in the weight tensor examples standalone usage initializer lecunnormal values initializershape2 2 usage in a keras layer initializer lecunnormal layer dense3 kernelinitializerinitializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference klambauer et al 2017https arxiv orgabs1706 02515 lecun uniform initializer draws samples from a uniform distribution within limit limit where limit sqrt3 fanin fanin is the number of input units in the weight tensor examples standalone usage initializer lecununiform values initializershape2 2 usage in a keras layer initializer lecununiform layer dense3 kernelinitializerinitializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference klambauer et al 2017https arxiv orgabs1706 02515 he normal initializer it draws samples from a truncated normal distribution centered on 0 with stddev sqrt2 fanin where fanin is the number of input units in the weight tensor examples standalone usage initializer henormal values initializershape2 2 usage in a keras layer initializer henormal layer dense3 kernelinitializerinitializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference he et al 2015https arxiv orgabs1502 01852 he uniform variance scaling initializer draws samples from a uniform distribution within limit limit where limit sqrt6 fanin fanin is the number of input units in the weight tensor examples standalone usage initializer heuniform values initializershape2 2 usage in a keras layer initializer heuniform layer dense3 kernelinitializerinitializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference he et al 2015https arxiv orgabs1502 01852 computes the number of input and output units for a weight shape args shape integer shape tuple returns a tuple of integer scalars fanin fanout assuming convolution kernels 2d 3d or more kernel shape inputdepth depth initializer that generates an orthogonal matrix if the shape of the tensor to initialize is twodimensional it is initialized with an orthogonal matrix obtained from the qr decomposition of a matrix of random numbers drawn from a normal distribution if the matrix has fewer rows than columns then the output will have orthogonal rows otherwise the output will have orthogonal columns if the shape of the tensor to initialize is more than twodimensional a matrix of shape shape0 shapen 2 shapen 1 is initialized where n is the length of the shape vector the matrix is subsequently reshaped to give a tensor of the desired shape examples standalone usage initializer keras initializers orthogonal values initializershape2 2 usage in a keras layer initializer keras initializers orthogonal layer keras layers dense3 kernelinitializerinitializer args gain multiplicative factor to apply to the orthogonal matrix seed a python integer used to make the behavior of the initializer deterministic reference saxe et al 2014https openreview netforum idwzzwkptdf9c flatten the input shape with the last dimension remaining its original shape so it works for conv2d generate a random matrix compute the qr factorization make q uniform random normal initializer draws samples from a normal distribution for given parameters examples standalone usage initializer randomnormal mean 0 0 stddev 1 0 values initializer shape 2 2 usage in a keras layer initializer randomnormal mean 0 0 stddev 1 0 layer dense 3 kernel_initializer initializer args mean a python scalar or a scalar keras tensor mean of the random values to generate stddev a python scalar or a scalar keras tensor standard deviation of the random values to generate seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator initializer that generates a truncated normal distribution the values generated are similar to values from a randomnormal initializer except that values more than two standard deviations from the mean are discarded and re drawn examples standalone usage initializer truncatednormal mean 0 stddev 1 values initializer shape 2 2 usage in a keras layer initializer truncatednormal mean 0 stddev 1 layer dense 3 kernel_initializer initializer args mean a python scalar or a scalar keras tensor mean of the random values to generate stddev a python scalar or a scalar keras tensor standard deviation of the random values to generate seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator random uniform initializer draws samples from a uniform distribution for given parameters examples standalone usage initializer randomuniform minval 0 0 maxval 1 0 values initializer shape 2 2 usage in a keras layer initializer randomuniform minval 0 0 maxval 1 0 layer dense 3 kernel_initializer initializer args minval a python scalar or a scalar keras tensor lower bound of the range of random values to generate inclusive maxval a python scalar or a scalar keras tensor upper bound of the range of random values to generate exclusive seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator initializer that adapts its scale to the shape of its input tensors with distribution truncated_normal or untruncated_normal samples are drawn from a truncated untruncated normal distribution with a mean of zero and a standard deviation after truncation if used stddev sqrt scale n where n is number of input units in the weight tensor if mode fan_in number of output units if mode fan_out average of the numbers of input and output units if mode fan_avg with distribution uniform samples are drawn from a uniform distribution within limit limit where limit sqrt 3 scale n examples standalone usage initializer variancescaling scale 0 1 mode fan_in distribution uniform values initializer shape 2 2 usage in a keras layer initializer variancescaling scale 0 1 mode fan_in distribution uniform layer dense 3 kernel_initializer initializer args scale scaling factor positive float mode one of fan_in fan_out fan_avg distribution random distribution to use one of truncated_normal untruncated_normal or uniform seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator the glorot uniform initializer also called xavier uniform initializer draws samples from a uniform distribution within limit limit where limit sqrt 6 fan_in fan_out fan_in is the number of input units in the weight tensor and fan_out is the number of output units examples standalone usage initializer glorotuniform values initializer shape 2 2 usage in a keras layer initializer glorotuniform layer dense 3 kernel_initializer initializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference glorot et al 2010 http proceedings mlr press v9 glorot10a html the glorot normal initializer also called xavier normal initializer draws samples from a truncated normal distribution centered on 0 with stddev sqrt 2 fan_in fan_out where fan_in is the number of input units in the weight tensor and fan_out is the number of output units in the weight tensor examples standalone usage initializer glorotnormal values initializer shape 2 2 usage in a keras layer initializer glorotnormal layer dense 3 kernel_initializer initializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference glorot et al 2010 http proceedings mlr press v9 glorot10a html lecun normal initializer initializers allow you to pre specify an initialization strategy encoded in the initializer object without knowing the shape and dtype of the variable being initialized draws samples from a truncated normal distribution centered on 0 with stddev sqrt 1 fan_in where fan_in is the number of input units in the weight tensor examples standalone usage initializer lecunnormal values initializer shape 2 2 usage in a keras layer initializer lecunnormal layer dense 3 kernel_initializer initializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference klambauer et al 2017 https arxiv org abs 1706 02515 lecun uniform initializer draws samples from a uniform distribution within limit limit where limit sqrt 3 fan_in fan_in is the number of input units in the weight tensor examples standalone usage initializer lecununiform values initializer shape 2 2 usage in a keras layer initializer lecununiform layer dense 3 kernel_initializer initializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference klambauer et al 2017 https arxiv org abs 1706 02515 he normal initializer it draws samples from a truncated normal distribution centered on 0 with stddev sqrt 2 fan_in where fan_in is the number of input units in the weight tensor examples standalone usage initializer henormal values initializer shape 2 2 usage in a keras layer initializer henormal layer dense 3 kernel_initializer initializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference he et al 2015 https arxiv org abs 1502 01852 he uniform variance scaling initializer draws samples from a uniform distribution within limit limit where limit sqrt 6 fan_in fan_in is the number of input units in the weight tensor examples standalone usage initializer heuniform values initializer shape 2 2 usage in a keras layer initializer heuniform layer dense 3 kernel_initializer initializer args seed a python integer or instance of keras backend seedgenerator used to make the behavior of the initializer deterministic note that an initializer seeded with an integer or none unseeded will produce the same random values across multiple calls to get different random values across multiple calls use as seed an instance of keras backend seedgenerator reference he et al 2015 https arxiv org abs 1502 01852 computes the number of input and output units for a weight shape args shape integer shape tuple returns a tuple of integer scalars fan_in fan_out just to avoid errors for constants assuming convolution kernels 2d 3d or more kernel shape input_depth depth initializer that generates an orthogonal matrix if the shape of the tensor to initialize is two dimensional it is initialized with an orthogonal matrix obtained from the qr decomposition of a matrix of random numbers drawn from a normal distribution if the matrix has fewer rows than columns then the output will have orthogonal rows otherwise the output will have orthogonal columns if the shape of the tensor to initialize is more than two dimensional a matrix of shape shape 0 shape n 2 shape n 1 is initialized where n is the length of the shape vector the matrix is subsequently reshaped to give a tensor of the desired shape examples standalone usage initializer keras initializers orthogonal values initializer shape 2 2 usage in a keras layer initializer keras initializers orthogonal layer keras layers dense 3 kernel_initializer initializer args gain multiplicative factor to apply to the orthogonal matrix seed a python integer used to make the behavior of the initializer deterministic reference saxe et al 2014 https openreview net forum id _wzzwkptdf_9c flatten the input shape with the last dimension remaining its original shape so it works for conv2d generate a random matrix compute the qr factorization make q uniform
import math

from keras import ops
from keras.api_export import keras_export
from keras.backend import random
from keras.initializers.initializer import Initializer
from keras.saving import serialization_lib


@keras_export(
    [
        "keras.initializers.RandomNormal",
        "keras.initializers.random_normal",
    ]
)
class RandomNormal(Initializer):
    

    def __init__(self, mean=0.0, stddev=0.05, seed=None):
        self.mean = mean
        self.stddev = stddev
        self._init_seed = seed
        self.seed = seed or random.make_default_seed()
        super().__init__()

    def __call__(self, shape, dtype=None):
        return random.normal(
            shape=shape,
            mean=self.mean,
            stddev=self.stddev,
            seed=self.seed,
            dtype=dtype,
        )

    def get_config(self):
        seed_config = serialization_lib.serialize_keras_object(self._init_seed)
        return {"mean": self.mean, "stddev": self.stddev, "seed": seed_config}


@keras_export(
    [
        "keras.initializers.TruncatedNormal",
        "keras.initializers.truncated_normal",
    ]
)
class TruncatedNormal(Initializer):
    

    def __init__(self, mean=0.0, stddev=0.05, seed=None):
        self.mean = mean
        self.stddev = stddev
        self._init_seed = seed
        self.seed = seed or random.make_default_seed()
        super().__init__()

    def __call__(self, shape, dtype=None):
        return random.truncated_normal(
            shape=shape,
            mean=self.mean,
            stddev=self.stddev,
            seed=self.seed,
            dtype=dtype,
        )

    def get_config(self):
        seed_config = serialization_lib.serialize_keras_object(self._init_seed)
        return {"mean": self.mean, "stddev": self.stddev, "seed": seed_config}


@keras_export(
    [
        "keras.initializers.RandomUniform",
        "keras.initializers.random_uniform",
    ]
)
class RandomUniform(Initializer):
    

    def __init__(self, minval=-0.05, maxval=0.05, seed=None):
        self.minval = minval
        self.maxval = maxval
        self._init_seed = seed
        self.seed = seed or random.make_default_seed()
        super().__init__()

    def __call__(self, shape, dtype=None):
        return random.uniform(
            shape=shape,
            minval=self.minval,
            maxval=self.maxval,
            seed=self.seed,
            dtype=dtype,
        )

    def get_config(self):
        seed_config = serialization_lib.serialize_keras_object(self._init_seed)
        return {
            "minval": self.minval,
            "maxval": self.maxval,
            "seed": seed_config,
        }


@keras_export(
    [
        "keras.initializers.VarianceScaling",
        "keras.initializers.variance_scaling",
    ]
)
class VarianceScaling(Initializer):
    

    def __init__(
        self,
        scale=1.0,
        mode="fan_in",
        distribution="truncated_normal",
        seed=None,
    ):
        if scale <= 0.0:
            raise ValueError(
                "Argument `scale` must be positive float. "
                f"Received: scale={scale}"
            )
        allowed_modes = {"fan_in", "fan_out", "fan_avg"}
        if mode not in allowed_modes:
            raise ValueError(
                f"Invalid `mode` argument: {mode}. "
                f"Please use one of {allowed_modes}"
            )
        distribution = distribution.lower()
        if distribution == "normal":
            distribution = "truncated_normal"
        allowed_distributions = {
            "uniform",
            "truncated_normal",
            "untruncated_normal",
        }
        if distribution not in allowed_distributions:
            raise ValueError(
                f"Invalid `distribution` argument: {distribution}."
                f"Please use one of {allowed_distributions}"
            )
        self.scale = scale
        self.mode = mode
        self.distribution = distribution
        self._init_seed = seed
        self.seed = seed or random.make_default_seed()

    def __call__(self, shape, dtype=None):
        scale = self.scale
        fan_in, fan_out = compute_fans(shape)
        if self.mode == "fan_in":
            scale /= max(1.0, fan_in)
        elif self.mode == "fan_out":
            scale /= max(1.0, fan_out)
        else:
            scale /= max(1.0, (fan_in + fan_out) / 2.0)
        if self.distribution == "truncated_normal":
            stddev = math.sqrt(scale) / 0.87962566103423978
            return random.truncated_normal(
                shape, mean=0.0, stddev=stddev, dtype=dtype, seed=self.seed
            )
        elif self.distribution == "untruncated_normal":
            stddev = math.sqrt(scale)
            return random.normal(
                shape, mean=0.0, stddev=stddev, dtype=dtype, seed=self.seed
            )
        else:
            limit = math.sqrt(3.0 * scale)
            return random.uniform(
                shape, minval=-limit, maxval=limit, dtype=dtype, seed=self.seed
            )

    def get_config(self):
        seed_config = serialization_lib.serialize_keras_object(self._init_seed)
        return {
            "scale": self.scale,
            "mode": self.mode,
            "distribution": self.distribution,
            "seed": seed_config,
        }


@keras_export(
    [
        "keras.initializers.GlorotUniform",
        "keras.initializers.glorot_uniform",
    ]
)
class GlorotUniform(VarianceScaling):
    

    def __init__(self, seed=None):
        super().__init__(
            scale=1.0, mode="fan_avg", distribution="uniform", seed=seed
        )

    def get_config(self):
        return {
            "seed": serialization_lib.serialize_keras_object(self._init_seed)
        }


@keras_export(
    [
        "keras.initializers.GlorotNormal",
        "keras.initializers.glorot_normal",
    ]
)
class GlorotNormal(VarianceScaling):
    

    def __init__(self, seed=None):
        super().__init__(
            scale=1.0,
            mode="fan_avg",
            distribution="truncated_normal",
            seed=seed,
        )

    def get_config(self):
        return {
            "seed": serialization_lib.serialize_keras_object(self._init_seed)
        }


@keras_export(
    [
        "keras.initializers.LecunNormal",
        "keras.initializers.lecun_normal",
    ]
)
class LecunNormal(VarianceScaling):
    

    def __init__(self, seed=None):
        super().__init__(
            scale=1.0, mode="fan_in", distribution="truncated_normal", seed=seed
        )

    def get_config(self):
        return {
            "seed": serialization_lib.serialize_keras_object(self._init_seed)
        }


@keras_export(
    [
        "keras.initializers.LecunUniform",
        "keras.initializers.lecun_uniform",
    ]
)
class LecunUniform(VarianceScaling):
    

    def __init__(self, seed=None):
        super().__init__(
            scale=1.0, mode="fan_in", distribution="uniform", seed=seed
        )

    def get_config(self):
        return {
            "seed": serialization_lib.serialize_keras_object(self._init_seed)
        }


@keras_export(["keras.initializers.HeNormal", "keras.initializers.he_normal"])
class HeNormal(VarianceScaling):
    

    def __init__(self, seed=None):
        super().__init__(
            scale=2.0, mode="fan_in", distribution="truncated_normal", seed=seed
        )

    def get_config(self):
        return {
            "seed": serialization_lib.serialize_keras_object(self._init_seed)
        }


@keras_export(["keras.initializers.HeUniform", "keras.initializers.he_uniform"])
class HeUniform(VarianceScaling):
    

    def __init__(self, seed=None):
        super().__init__(
            scale=2.0, mode="fan_in", distribution="uniform", seed=seed
        )

    def get_config(self):
        return {
            "seed": serialization_lib.serialize_keras_object(self._init_seed)
        }


def compute_fans(shape):
    
    shape = tuple(shape)
    if len(shape) < 1:  
        fan_in = fan_out = 1
    elif len(shape) == 1:
        fan_in = fan_out = shape[0]
    elif len(shape) == 2:
        fan_in = shape[0]
        fan_out = shape[1]
    else:
        
        
        receptive_field_size = 1
        for dim in shape[:-2]:
            receptive_field_size *= dim
        fan_in = shape[-2] * receptive_field_size
        fan_out = shape[-1] * receptive_field_size
    return int(fan_in), int(fan_out)


@keras_export(
    [
        "keras.initializers.OrthogonalInitializer",
        "keras.initializers.Orthogonal",
        "keras.initializers.orthogonal",
    ]
)
class OrthogonalInitializer(Initializer):
    

    def __init__(self, gain=1.0, seed=None):
        self.gain = gain
        self._init_seed = seed
        self.seed = seed or random.make_default_seed()

    def __call__(self, shape, dtype=None):
        if len(shape) < 2:
            raise ValueError(
                "The tensor to initialize must be "
                "at least two-dimensional. Received: "
                f"shape={shape} of rank {len(shape)}."
            )

        
        
        num_rows = 1
        for dim in shape[:-1]:
            num_rows *= dim
        num_cols = shape[-1]
        flat_shape = (max(num_cols, num_rows), min(num_cols, num_rows))

        
        a = random.normal(flat_shape, seed=self.seed, dtype=dtype)
        
        q, r = ops.qr(a)
        
        d = ops.diag(r)
        q *= ops.sign(d)
        if num_rows < num_cols:
            q = ops.transpose(q)
        return self.gain * ops.reshape(q, shape)

    def get_config(self):
        seed_config = serialization_lib.serialize_keras_object(self._init_seed)
        return {"gain": self.gain, "seed": seed_config}