base class for weight constraints a constraint instance works like a stateless function users who subclass this class should override the call method which takes a single weight parameter and return a projected version of that parameter e g normalized or clipped constraints can be used with various keras layers via the kernelconstraint or biasconstraint arguments here s a simple example of a nonnegative weight constraint class nonnegativekeras constraints constraint def callself w return w ops castops greaterequalw 0 dtypew dtype weight ops converttotensor1 0 1 0 nonnegativeweight 0 1 usage in a layer keras layers dense4 kernelconstraintnonnegative applies the constraint to the input weight variable by default the inputs weight variable is not modified users should override this method to implement their own projection function args w input weight variable returns projected variable by default returns unmodified inputs returns a python dict of the object config a constraint config is a python dictionary jsonserializable that can be used to reinstantiate the same object returns python dict containing the configuration of the constraint object instantiates a weight constraint from a configuration dictionary example python constraint unitnorm config constraint getconfig constraint unitnorm fromconfigconfig args config a python dictionary the output of getconfig returns a keras constraints constraint instance maxnorm weight constraint constrains the weights incident to each hidden unit to have a norm less than or equal to a desired value also available via the shortcut function keras constraints maxnorm args maxvalue the maximum norm value for the incoming weights axis integer axis along which to calculate weight norms for instance in a dense layer the weight matrix has shape inputdim outputdim set axis to 0 to constrain each weight vector of length inputdim in a conv2d layer with dataformatchannelslast the weight tensor has shape rows cols inputdepth outputdepth set axis to 0 1 2 to constrain the weights of each filter tensor of size rows cols inputdepth constrains the weights to be nonnegative def callself w w backend converttotensorw return w ops castops greaterequalw 0 0 dtypew dtype kerasexportkeras constraints unitnorm keras constraints unitnorm class unitnormconstraint def initself axis0 self axis axis def callself w w backend converttotensorw return w backend epsilon ops sqrtops sumops squarew axisself axis keepdimstrue def getconfigself return axis self axis kerasexport keras constraints minmaxnorm keras constraints minmaxnorm class minmaxnormconstraint def initself minvalue0 0 maxvalue1 0 rate1 0 axis0 self minvalue minvalue self maxvalue maxvalue self rate rate self axis axis def callself w w backend converttotensorw norms ops sqrtops sumops squarew axisself axis keepdimstrue desired self rate ops clipnorms self minvalue self maxvalue 1 self rate norms return w desired backend epsilon norms def getconfigself return minvalue self minvalue maxvalue self maxvalue rate self rate axis self axis base class for weight constraints a constraint instance works like a stateless function users who subclass this class should override the __call__ method which takes a single weight parameter and return a projected version of that parameter e g normalized or clipped constraints can be used with various keras layers via the kernel_constraint or bias_constraint arguments here s a simple example of a non negative weight constraint class nonnegative keras constraints constraint def __call__ self w return w ops cast ops greater_equal w 0 dtype w dtype weight ops convert_to_tensor 1 0 1 0 nonnegative weight 0 1 usage in a layer keras layers dense 4 kernel_constraint nonnegative applies the constraint to the input weight variable by default the inputs weight variable is not modified users should override this method to implement their own projection function args w input weight variable returns projected variable by default returns unmodified inputs returns a python dict of the object config a constraint config is a python dictionary json serializable that can be used to reinstantiate the same object returns python dict containing the configuration of the constraint object instantiates a weight constraint from a configuration dictionary example python constraint unitnorm config constraint get_config constraint unitnorm from_config config args config a python dictionary the output of get_config returns a keras constraints constraint instance maxnorm weight constraint constrains the weights incident to each hidden unit to have a norm less than or equal to a desired value also available via the shortcut function keras constraints max_norm args max_value the maximum norm value for the incoming weights axis integer axis along which to calculate weight norms for instance in a dense layer the weight matrix has shape input_dim output_dim set axis to 0 to constrain each weight vector of length input_dim in a conv2d layer with data_format channels_last the weight tensor has shape rows cols input_depth output_depth set axis to 0 1 2 to constrain the weights of each filter tensor of size rows cols input_depth constrains the weights to be non negative constrains the weights incident to each hidden unit to have unit norm args axis integer axis along which to calculate weight norms for instance in a dense layer the weight matrix has shape input_dim output_dim set axis to 0 to constrain each weight vector of length input_dim in a conv2d layer with data_format channels_last the weight tensor has shape rows cols input_depth output_depth set axis to 0 1 2 to constrain the weights of each filter tensor of size rows cols input_depth minmaxnorm weight constraint constrains the weights incident to each hidden unit to have the norm between a lower bound and an upper bound args min_value the minimum norm for the incoming weights max_value the maximum norm for the incoming weights rate rate for enforcing the constraint weights will be rescaled to yield 1 rate norm rate norm clip min_value max_value effectively this means that rate 1 0 stands for strict enforcement of the constraint while rate 1 0 means that weights will be rescaled at each step to slowly move towards a value inside the desired interval axis integer axis along which to calculate weight norms for instance in a dense layer the weight matrix has shape input_dim output_dim set axis to 0 to constrain each weight vector of length input_dim in a conv2d layer with data_format channels_last the weight tensor has shape rows cols input_depth output_depth set axis to 0 1 2 to constrain the weights of each filter tensor of size rows cols input_depth
from keras import backend
from keras import ops
from keras.api_export import keras_export


@keras_export("keras.constraints.Constraint")
class Constraint:
    

    def __call__(self, w):
        
        return w

    def get_config(self):
        
        return {}

    @classmethod
    def from_config(cls, config):
        
        return cls(**config)


@keras_export(["keras.constraints.MaxNorm", "keras.constraints.max_norm"])
class MaxNorm(Constraint):
    

    def __init__(self, max_value=2, axis=0):
        self.max_value = max_value
        self.axis = axis

    def __call__(self, w):
        w = backend.convert_to_tensor(w)
        norms = ops.sqrt(ops.sum(ops.square(w), axis=self.axis, keepdims=True))
        desired = ops.clip(norms, 0, self.max_value)
        return w * (desired / (backend.epsilon() + norms))

    def get_config(self):
        return {"max_value": self.max_value, "axis": self.axis}


@keras_export(["keras.constraints.NonNeg", "keras.constraints.non_neg"])
class NonNeg(Constraint):
    

    def __call__(self, w):
        w = backend.convert_to_tensor(w)
        return w * ops.cast(ops.greater_equal(w, 0.0), dtype=w.dtype)


@keras_export(["keras.constraints.UnitNorm", "keras.constraints.unit_norm"])
class UnitNorm(Constraint):
    

    def __init__(self, axis=0):
        self.axis = axis

    def __call__(self, w):
        w = backend.convert_to_tensor(w)
        return w / (
            backend.epsilon()
            + ops.sqrt(ops.sum(ops.square(w), axis=self.axis, keepdims=True))
        )

    def get_config(self):
        return {"axis": self.axis}


@keras_export(
    ["keras.constraints.MinMaxNorm", "keras.constraints.min_max_norm"]
)
class MinMaxNorm(Constraint):
    

    def __init__(self, min_value=0.0, max_value=1.0, rate=1.0, axis=0):
        self.min_value = min_value
        self.max_value = max_value
        self.rate = rate
        self.axis = axis

    def __call__(self, w):
        w = backend.convert_to_tensor(w)
        norms = ops.sqrt(ops.sum(ops.square(w), axis=self.axis, keepdims=True))
        desired = (
            self.rate * ops.clip(norms, self.min_value, self.max_value)
            + (1 - self.rate) * norms
        )
        return w * (desired / (backend.epsilon() + norms))

    def get_config(self):
        return {
            "min_value": self.min_value,
            "max_value": self.max_value,
            "rate": self.rate,
            "axis": self.axis,
        }