<jupyter_start><jupyter_text>Activity 1 - ReLU NeuronsIn this activity we will investigate the Rectified Linear Unit (ReLU) activation function and the effect the tunable weights have in modifying the output of ReLU neurons.Import numpy and matplotlib<jupyter_code>import numpy as np
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Define the ReLU activation function as a Python function<jupyter_code>def relu(x):
    return np.max((0, x))<jupyter_output><empty_output><jupyter_text>Define the inputs ($x$) and tunable weights ($\Theta$) for the neuron.  In this example the inputs ($x$) will be 100 numbers linearly spaced between -5 and 5, initially $\Theta=1$.<jupyter_code>theta = 1
x = np.linspace(-5, 5, 100)
x<jupyter_output><empty_output><jupyter_text>Compute the outputs ($y$)<jupyter_code>y = [relu(_x * theta) for _x in x]<jupyter_output><empty_output><jupyter_text>Plot the output of the neuron vs the input<jupyter_code>fig = plt.figure(figsize=(10, 7))
ax = fig.add_subplot(111)

ax.plot(x, y)
ax.set_xlabel('$x$', fontsize=22);
ax.set_ylabel('$h(x\Theta)$', fontsize=22);
ax.spines['left'].set_position(('data', 0));
ax.spines['top'].set_visible(False);
ax.spines['right'].set_visible(False);
ax.tick_params(axis='both', which='major', labelsize=22)<jupyter_output><empty_output><jupyter_text>Now set $\Theta=5$, recomupte and store the output of the neuron<jupyter_code>theta = 5
y_2 = [relu(_x * theta) for _x in x]<jupyter_output><empty_output><jupyter_text>Now set $\Theta=0.2$, recomupte and store the output of the neuron<jupyter_code>theta = 0.2
y_3 = [relu(_x * theta) for _x in x]<jupyter_output><empty_output><jupyter_text>Plot the three different output curves of the neuron ($\Theta=1, \Theta=5, \Theta=0.2$) on the one graph.<jupyter_code>fig = plt.figure(figsize=(10, 7))
ax = fig.add_subplot(111)

ax.plot(x, y, label='$\Theta=1$');
ax.plot(x, y_2, label='$\Theta=5$', linestyle=':');
ax.plot(x, y_3, label='$\Theta=0.2$', linestyle='--');
ax.set_xlabel('$x\Theta$', fontsize=22);
ax.set_ylabel('$h(x\Theta)$', fontsize=22);
ax.spines['left'].set_position(('data', 0));
ax.spines['top'].set_visible(False);
ax.spines['right'].set_visible(False);
ax.tick_params(axis='both', which='major', labelsize=22);
ax.legend(fontsize=22);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16371490280777537, 'fraction_numerical': 0.02591792656587473, 'mean_word_length': 7.391304347826087, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '32090', 'license': 'MIT', 'n_tokens_mistral': 916, 'n_tokens_neox': 866, 'n_words': 198, 'path': 'Lesson05/Activity08/Activity08.ipynb', 'repo_name': 'TrainingByPackt/Unsupervised-Learning-with-Python', 'size': '2315'}