<jupyter_start><jupyter_text>Losses and metrics are conceptually not the same thing: **losses are used by Gradient Descent to train a model**, so they must be differentiable (at least where they are evaluated) and their gradients should not be 0 everywhere. Plus, it’s okay if they are not easily interpretable by humans (e.g. cross-entropy). In contrast, metrics are used to evaluate a model, they must be more easily interpretable, and they can be nondifferentiable or have 0 gradients everywhere (e.g., accuracy). That said, in most cases, defining a custom metric function is exactly the same asdefining a custom loss function. In fact, we could even use the Huber loss function wecreated earlier in **22_implementing_custom_loss_and_saving_the_model.ipynb**, it would work just fine (and persistence would also work the same way, in this case only saving the name of the function, "huber_floss" ):```model.compile(loss="mse", optimizer="nadam", metrics=[create_huber(2.0)])```<jupyter_code>precision = tf.keras.metrics.Precision()
precision([0, 1, 1, 1, 0, 1, 0, 1], [1, 1, 0, 1, 0, 1, 0, 1])
precision([0, 1, 0, 0, 1, 0, 1, 1], [1, 0, 1, 1, 0, 0, 0, 0])
precision.result()
precision.variables
precision.reset_states()
precision.variables<jupyter_output><empty_output><jupyter_text>If you need to create such a streaming metric, you can just create a subclass of thekeras.metrics.Metric class. Here is a simple example that keeps track of the totalHuber loss and the number of instances seen so far. When asked for the result, itreturns the ratio, which is simply the mean Huber loss:<jupyter_code>def create_huber(threshold=1.0):
    def huber_fn(y_true, y_pred):
        error = y_true - y_pred
        is_small_error = tf.abs(error) < threshold
        squared_loss = tf.square(error) / 2
        linear_loss = threshold * tf.abs(error) - threshold**2 / 2
        return tf.where(is_small_error, squared_loss, linear_loss)
    return huber_fn
class HuberMetric(tf.keras.metrics.Metric):
    def __init__(self, threshold=1.0, **kwargs):
        super().__init__(**kwargs) # handles base args (e.g., dtype)
        self.threshold = threshold
        self.huber_fn = create_huber(threshold)
        self.total = self.add_weight("total", initializer="zeros")
        self.count = self.add_weight("count", initializer="zeros")
    def update_state(self, y_true, y_pred, sample_weight=None):
        metric = self.huber_fn(y_true, y_pred)
        self.total.assign_add(tf.reduce_sum(metric))
        self.count.assign_add(tf.cast(tf.size(y_true), tf.float32))
    def result(self):
        return self.total / self.count
    def get_config(self):
        base_config = super().get_config()
        return {**base_config, "threshold": self.threshold}<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10571736785329018, 'fraction_numerical': 0.01690039554117224, 'mean_word_length': 4.278937381404175, 'pattern_counts': {'":': 1, '<': 10, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '263919', 'license': 'MIT', 'n_tokens_mistral': 902, 'n_tokens_neox': 846, 'n_words': 297, 'path': '24_custom_metrics.ipynb', 'repo_name': 'renjithbaby23/tf2.0_examples', 'size': '2781'}