<jupyter_start><jupyter_text>[![Open in Colab](https://img.shields.io/static/v1?label=&message=Open%20in%20Colab&labelColor=grey&color=blue&logo=google-colab)](https://colab.research.google.com/github/theaveas/DeepLearning/blob/main/NNFS/02_nnfs_coding_our_first_neurons.ipynbscrollTo=00aa98bb-97e9-4872-b87a-90f12687587f)  Coding Our First Neurons- We assume that we all have a basic of programing here.- In this Notebook we focusing on using `python` also `numpy` for some vectorizaton and `matplotlib` for some visualizations.<jupyter_code>import platform
import numpy as np
import matplotlib
import matplotlib.pyplot as plt

print(platform.python_version())
print(np.__version__)
print(matplotlib. __version__)

#python version 3.9.7
#numpy version 1.21.2
#matplotlib version 3.5.0<jupyter_output><empty_output><jupyter_text>A Single Neuron- As we discussed in previous notebook. - `feature` a group of features makes up a feature set (represented as vectors/arrays), and the values of a feature set can be reffed to as a `sample` (training example)- Let's say we have a single neuron, and there inputs to this neuron<jupyter_code># single neuron with three input features
inputs = [1, 2, 3]<jupyter_output><empty_output><jupyter_text>- Each input need to have parameters, knob or **weight** associated with it<jupyter_code># initialized weight
weights = [0.2, 0.8, -0.5]<jupyter_output><empty_output><jupyter_text>- Finally we also need the bias, At the moment, we're modeling a single neuron with three input. So we only need one bias<jupyter_code># initialized bias
bias = 2<jupyter_output><empty_output><jupyter_text>- Remember we talk about the output function `yhat`?- `yhat` = **sum(weights * inputs) + bias<jupyter_code># calculate the output layer
output = (inputs[0] * weights[0] +
          inputs[1] * weights[1] +
          inputs[2] * weights[2] + bias)

print(f'output layer of this single neuron is {output}')<jupyter_output><empty_output><jupyter_text>- What might we need to change if we have 4 inputs, rather than 3<jupyter_code># input features one neuron with four features
inputs = [1, 2, 3, 2.5]

# weight associate to the inputs 
weights = [0.2, 0.8, -0.5, 1.0]

# one neuron = one bias
bias = 2.0

output = (inputs[0] * weights[0] +
          inputs[1] * weights[1] +
          inputs[2] * weights[2] +
          inputs[3] * weights[3] + bias)

print(f'output layer of this single neuron is {output}')<jupyter_output><empty_output><jupyter_text>Layers of Neurons- Same things but with multiple neurons- Let's say we have a scenario with three neurons in layer and 4 inputs<jupyter_code># input features one neuron with four features
inputs = [1, 2, 3, 2.5]

# three weight associate to the inputs 
weights1 = [0.2, 0.8, -0.5, 1.0]
weights2 = [0.5, -0.91, 0.26, -0.5]
weights3 = [-0.26, -0.27, 0.17, 0.87]

# because of we have three neurons that right there will be three bias
bias1 = 2
bias2 = 3
bias3 = 0.5

output = [
    # neuron 1
    (inputs[0] * weights1[0] +
     inputs[1] * weights1[1] +
     inputs[2] * weights1[2] +
     inputs[3] * weights1[3] + bias1),
    # neuron 2
    (inputs[0] * weights2[0] +
     inputs[1] * weights2[1] +
     inputs[2] * weights2[2] +
     inputs[3] * weights2[3] + bias2),
    # neuron 3
    (inputs[0] * weights3[0] +
     inputs[1] * weights3[1] +
     inputs[2] * weights3[2] +
     inputs[3] * weights3[3] + bias3)]
    
print(f'output layer of this three neurons is {output}')<jupyter_output><empty_output><jupyter_text>- Let use for loop<jupyter_code># input features one neuron with four features
inputs = [1, 2, 3, 2.5]
weights = [[0.2, 0.8, -0.5, 1.0],
           [0.5, -0.91, 0.26, -0.5],
           [-0.26, -0.27, 0.17, 0.87]]
biases = [2, 3, 0.5]


layer_outputs = []
# for each neuron
for neuron_weights, neuron_bias in zip(weights, biases):
    # zeroed output of given neuron
    neuron_output = 0
    # for each input and weight to the neuron
    for neuron_input, neuron_weight in zip (inputs, neuron_weights):
        # apply output layer function ( weight * input_features) + bias
        neuron_output += neuron_input * neuron_weight
        
    layer_outputs.append(neuron_output + neuron_bias)
    
print(f'output layers of this three neurons is {layer_outputs}')<jupyter_output><empty_output><jupyter_text>Tensors Arrays Vectors- In Deep learning, Because of the dataset is huge, using for loop is not going to work.- We use `Tensors`, `Arrays`, `Vectors`, and `Matrixes` instead<jupyter_code># array or list
l = [1, 5, 6, 2]
# a list of lists
lol = [[1, 5, 6, 2],
       [3, 4, 1, 3]]
# list of lists of lists
lolol = [[[1,5,6,2],
          [3,2,1,3]],
         [[5,2,1,2],
          [6,4,8,4]],
         [[2,8,5,3],
          [1,1,9,4]]]<jupyter_output><empty_output><jupyter_text>Dot Product and Vector Addition- To simplyfied the process we use `numpy`- if you not yet install numpy:    - !pip install numpy<jupyter_code># dot product using pure python
a = [1, 2, 3]
b = [2, 3, 4]

dot_product = a[0] * b[0] + a[1] * b[1] + a[2] * b[2]
print(dot_product)
# dot product using numpy
a = [1, 2, 3]
b = [2, 3, 4]

dot_product = np.dot(a, b)
print(dot_product)<jupyter_output><empty_output><jupyter_text>Single neuron with Numpy- You will be surprise how easy to calculate this function<jupyter_code># input features one neuron with four features
inputs = [1, 2, 3, 2.5]

weights = [0.2, 0.8, -0.5, 1.0]
bias = 2.0

output = np.dot(weights, inputs) + bias
print(f'output layer of this single neuron is {output}')<jupyter_output><empty_output><jupyter_text>A Layer of Neurons with Numpy<jupyter_code># input features one neuron with four features
inputs = np.array([1, 2, 3, 2.5])
weights = np.array([[0.2, 0.8, -0.5, 1.0],
           [0.5, -0.91, 0.26, -0.5],
           [-0.26, -0.27, 0.17, 0.87]])
biases = [2, 3, 0.5]

output = np.dot(weights, inputs) + bias
print(f'output layer of this single neuron is {output}')<jupyter_output><empty_output><jupyter_text>A Batch of Data- To train, neural netowork tend to recieve data in batch.<jupyter_code>batch = np.array([[1,5,6,2],
                  [3,2,1,3],
                  [5,2,1,2],
                  [6,4,8,4],
                  [2,8,5,3],
                  [1,1,9,4],
                  [6,6,0,4],
                  [8,7,6,4]])
print(batch.shape)
print('Simply as we have a 8 training example, and each sample have 4 features')
# input features one neuron with four features
inputs = np.array([[1, 2, 3, 2.5],
                   [2, 5, -1, 2],
                   [-1.5, 2.7, 3.3, -0.8]])
weights = np.array([[0.2, 0.8, -0.5, 1.0],
           [0.5, -0.91, 0.26, -0.5],
           [-0.26, -0.27, 0.17, 0.87]])
biases = [2, 3, 0.5]

output = np.dot(weights, inputs) + bias
print(f'output layer of this single neuron is {output}')<jupyter_output><empty_output><jupyter_text>- You might get **ValueError: shapes (3,4) and (3,4) not aligned: 4 (dim 1) != 3 (dim 0)**- This is the real pain in the ass for me when I start learning DeepLearning- To multiply matrix we need match the `Dimension`- `(a, n)` dot `(n, b)`<jupyter_code># input features one neuron with four features
inputs = np.array([[1, 2, 3, 2.5],
                   [2, 5, -1, 2],
                   [-1.5, 2.7, 3.3, -0.8]])
weights = np.array([[0.2, 0.8, -0.5, 1.0],
           [0.5, -0.91, 0.26, -0.5],
           [-0.26, -0.27, 0.17, 0.87]])
biases = [2, 3, 0.5]

# np.Transpose is the function allow you to swap columns to rows and rows to columns
output = np.dot(weights, inputs.T) + bias
print(f'output layer of this single neuron is \n{output}')<jupyter_output><empty_output><jupyter_text>[**Learn more about Linear algebra**](https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab) A Youtube playlist by **3blue1brown**<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14393842206542656, 'fraction_numerical': 0.059781911481719054, 'mean_word_length': 3.454857142857143, 'pattern_counts': {'":': 0, '<': 61, '<?xml version=': 0, '>': 61, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '543460', 'license': 'MIT', 'n_tokens_mistral': 3102, 'n_tokens_neox': 2867, 'n_words': 867, 'path': 'NNFS/02_nnfs_coding_our_first_neurons.ipynb', 'repo_name': 'theaveas/DeepLearning', 'size': '7795'}