<jupyter_start><jupyter_text>``ReLU`` Activation Function<jupyter_code>import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np<jupyter_output><empty_output><jupyter_text>`Relu` and TensorFlow<jupyter_code>inputs = tf.constant(np.arange(-10, 10), dtype=tf.float32)
inputs
outputs = tf.keras.activations.relu(inputs)
outputs<jupyter_output><empty_output><jupyter_text>> This activations turns all numbers that aree less than ``0`` to `zeros` and others remains unchanged.<jupyter_code>outputs = tf.maximum(inputs, 0)
outputs<jupyter_output><empty_output><jupyter_text>> The output is still the same. Let's create a class `Activation` and implemant the `math` behind `ReLU`<jupyter_code>class Activations:
    def __init__(self):
        pass
    
    def relu(self, x):
        return tf.maximum(x, 0)
    
    def relu_(self, x):
        return tf.constant([0 if i< 0 else i for i in x.numpy()], dtype=x.dtype)
activations = Activations()
output = activations.relu(inputs)
output<jupyter_output><empty_output><jupyter_text>> Output is still the same<jupyter_code>outputs = activations.relu_(inputs)
outputs<jupyter_output><empty_output><jupyter_text>> The output is still the same. Visualisation of the `ReLU` activation function<jupyter_code>inputs, outputs
plt.plot(inputs, outputs)
plt.title('ReLU')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11315209404849376, 'fraction_numerical': 0.008082292432035268, 'mean_word_length': 6.32258064516129, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '270185', 'license': 'MIT', 'n_tokens_mistral': 488, 'n_tokens_neox': 468, 'n_words': 113, 'path': '02_Activation_Functions/01_ReLU/.ipynb_checkpoints/ReLu-checkpoint.ipynb', 'repo_name': 'CrispenGari/keras-api', 'size': '1361'}