<jupyter_start><jupyter_text>(Real_Non_Linear_Neural_Network)= Chapter 7 -- Real (Non-linear) Neural Network So in the previous example, we derived the gradients for a two layers neural network. This is to find the straight line that bisects the two groups in figure 7.1 in the introduction. However, in reality, we often have the following groups: Figure 7.1 For data like this, a linear separator cannot satisfy our needs. The solution is to add another linear separator on top of the original linear separator. Figure 7.2 This is a classic three layers neural network. The layer at the left hand side is called the input layer; the layer at the right hand side is known as the output layer; the layer in between is called the hidden layer. The hidden layer is like a black-box that we cannot usually interpret by our instinct. We will dig in more details later. Figure 7.3 Figure 7.4 This is becoming something like a network finally. But the way we express the weights get more complicated. Here is how we define it by tradition: $$w_{ab}^{(n)}$$ (eq7_1) where $n$ means the $n^{th}$ layer in the neural net; $n$ = $1$ at the input layer. Suppose $n=1$ at the input layer, then $a$ and $b$ means that the weight is pointing from the $a^{th}$ neural in the second (hidden) layer to the $b^{th}$ neural in the input layer. This is going backwards (to the left). Figure 7.5 For example, the weights in the input layer in the figure 7.4 can be defined as follow $$W^{(1)}= \begin{bmatrix}w^{(1)}_{11} & w^{(1)}_{21} \\w^{(1)}_{12} & w^{(1)}_{22} \\w^{(1)}_{13} & w^{(1)}_{23}\end{bmatrix} = \begin{bmatrix}5 & 7 \\-2 & -3 \\-8 & 1\end{bmatrix} $$ (eq7_2) And the weights in the hidden layer can be defined as $$W^{(2)}= \begin{bmatrix}w^{(2)}_{11}  \\w^{(2)}_{12}  \\w^{(2)}_{13} \end{bmatrix} = \begin{bmatrix} 7 \\5 \\-6\end{bmatrix} $$ (eq7_3) In python, we can describe the core features of such network by defining a Network clas. Here's the code we use to initialize a Network object:<jupyter_code>import numpy as np

class Network(object):

    def __init__(self, sizes):
        self.num_layers = len(sizes)
        self.sizes = sizes
        self.biases = [np.random.randn(y, 1) for y in sizes[1:]]
        self.weights = [np.random.randn(y, x) 
                        for x, y in zip(sizes[:-1], sizes[1:])]<jupyter_output><empty_output><jupyter_text>In this code, the list sizes contains the number of neurons in the respective layers. So, for example, if we want to create a Network object with 2 neurons in the first layer, 3 neurons in the second layer, and 1 neuron in the final layer, we'd do this with the code:<jupyter_code>net = Network([2, 3, 1])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11420096403411198, 'fraction_numerical': 0.02632554690396737, 'mean_word_length': 4.014869888475836, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '16177', 'license': 'MIT', 'n_tokens_mistral': 829, 'n_tokens_neox': 778, 'n_words': 395, 'path': 'notebooks/e_extra/pytorch_image_filtering_ml/Chapter 7 -- Real (Non-linear) Neural Network.ipynb', 'repo_name': 'primer-computational-mathematics/book', 'size': '2697'}