<jupyter_start><jupyter_text>Exercise 1**Writing a simple neural network function**Given an input and weight as the input of the function, write a neural network function that return the prediction value as the result.The input is is a list of random numbers but with small difference between each numbers.<jupyter_code>weight = 0.1
def neural_network(input, weight):
  prediction = input * weight
  return prediction

number_of_items = [7.0, 7.2, 7.4, 7.8, 8.5]
input = number_of_items[0]
pred = neural_network(input, weight)
print(pred)<jupyter_output>0.7000000000000001<jupyter_text>Exercise 2**Writing a softmax function**1. Write a function that takes an input a list of numbers, and returns the list of values given by the softmax function.2. By using [keras](https://keras.io/activations/), write the softmax function.---$f(x_{i}) =\frac{e^{x_{i}}}{\sum_{j=0}^{k}{e^{x_{j}}}}    $where $i = 0,1,2,...k$<jupyter_code>#solution number 1
import numpy as np

def softmax(L):
  exponential_L = np.exp(L)
  sum_exponential_L = sum(exponential_L)
  result = []
  for i in exponential_L:
    result.append(i*1.0/ sum_exponential_L)
  return result

L = [5,6,7]

print(softmax(L))
#solution number 2

from keras.models import Sequential
from keras.layers import Activation, Dense

#create a model
model = Sequential()

#add the layer
model.add(Dense(64))
model.add(Activation('softmax'))<jupyter_output><empty_output><jupyter_text>Exercise 3**Writing a cross entropy function**1. Write a function that takes as input two list Y, P and returns the float corresponding to their cross-entropy.2. Write the cross-entropy by using [keras](https://keras.io/losses/).---$H(y, \hat{y}) = \sum_{i}{y_i}log{\frac{1}{\hat{y_i}}}=-\sum_{i}{y_i}log\hat{y_i}$<jupyter_code>#solution 1

import numpy as np
def cross_entropy(Y, P):
    y = np.float_(Y)
    p = np.float_(P)
    return -np.sum(y * np.log(p)+ (1 - y) * np.log(1 - p))
  
Y = [1,0,1,1]
P = [0.4, 0.6, 0.1, 0.5]

print(cross_entropy(Y,P))
#solution 2

model.compile(loss='binary_crossentropy', optimizer ='adam')<jupyter_output><empty_output><jupyter_text>Exercise 4**Creating a simple deep learning**This solution is referred from [Grokking Deep Learning Repo](https://github.com/iamtrask/Grokking-Deep-Learning/blob/master/Chapter6%20-%20Intro%20to%20Backpropagation%20-%20Building%20Your%20First%20DEEP%20Neural%20Network.ipynb).Imagine we have a traffic light (street lights):0 is red = stop1 is green = walkCalculate the prediction and training loss to predict the next color for the traffic light.<jupyter_code>import numpy as np
weights = np.array([0.5,0.48,-0.7])
alpha = 0.1

streetlights = np.array( [ [ 1, 0, 1 ],
                           [ 0, 1, 1 ],
                           [ 0, 0, 1 ],
                           [ 1, 1, 1 ],
                           [ 0, 1, 1 ],
                           [ 1, 0, 1 ] ] )

walk_vs_stop = np.array( [ 0, 1, 0, 1, 1, 0 ] )

input = streetlights[0] # [1,0,1]
goal_prediction = walk_vs_stop[0] # equals 0... i.e. "stop"

for iteration in range(30):
    prediction = input.dot(weights)
    error = (goal_prediction - prediction) ** 2
    delta = prediction - goal_prediction
    weights = weights - (alpha * (input * delta))	

    print("Error:" + str(error) + " Prediction:" + str(prediction))<jupyter_output>Error:0.03999999999999998 Prediction:-0.19999999999999996
Error:0.025599999999999973 Prediction:-0.15999999999999992
Error:0.01638399999999997 Prediction:-0.1279999999999999
Error:0.010485759999999964 Prediction:-0.10239999999999982
Error:0.006710886399999962 Prediction:-0.08191999999999977
Error:0.004294967295999976 Prediction:-0.06553599999999982
Error:0.002748779069439994 Prediction:-0.05242879999999994
Error:0.0017592186044416036 Prediction:-0.04194304000000004
Error:0.0011258999068426293 Prediction:-0.03355443200000008
Error:0.0007205759403792803 Prediction:-0.02684354560000002
Error:0.0004611686018427356 Prediction:-0.021474836479999926
Error:0.0002951479051793508 Prediction:-0.01717986918399994
Error:0.00018889465931478573 Prediction:-0.013743895347199997
Error:0.00012089258196146188 Prediction:-0.010995116277759953
Error:7.737125245533561e-05 Prediction:-0.008796093022207963
Error:4.951760157141604e-05 Prediction:-0.007036874417766459
Error:3.169126500570676e-05 Prediction:-0.0[...]
{'fraction_non_alphanumeric': 0.12781257248898167, 'fraction_numerical': 0.17536534446764093, 'mean_word_length': 4.528205128205128, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 14, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '466953', 'license': 'MIT', 'n_tokens_mistral': 2010, 'n_tokens_neox': 1539, 'n_words': 352, 'path': 'Lesson15/Lesson15.ipynb', 'repo_name': 'KeyBlades/AI_course', 'size': '4311'}