<jupyter_start><jupyter_text>Deep Learning with Keras Step-By-Step Keras is a powerful and easy-to-use free open source Python library for developingand evaluating deep learning models.It wraps the efficient numerical computation libraries Theano and TensorFlow andallows you to define and train neural network models in just a few lines of code.In this tutorial, you will discover how to create your first deep learning neuralnetwork model in Python using Keras.Kick-start your project with my new book Deep Learning With Python, includingstep-by-step tutorials and the Python source code files for all examples.There is not a lot of code required, but we are going to step over it slowly so thatyou will know how to create your own models in the future.The steps you are going to cover in this tutorial are as follows: Load Data. Define Keras Model. Compile Keras Model. Fit Keras Model. Evaluate Keras Model. Tie It All Together. Make PredictionsThis Keras tutorial has a few requirements: You have Python 2 or 3 installed and configured. You have SciPy (including NumPy) installed and configured. You have Keras and a backend (Theano or TensorFlow) installed and configured.  Using Moon Dataset<jupyter_code>np.random.seed(0)
X, y= make_moons(1000, noise= 0.20)
x_train, x_test, y_train, y_test= train_test_split(X, y, test_size= 0.20, random_state= 42)
plt.scatter(X[:,0], X[:,1], s= 40, c=y, cmap=plt.cm.Spectral)
print(X)<jupyter_output>[[ 2.08614399  0.57401401]
 [ 1.80460944  0.00695963]
 [-0.22711217  0.57390962]
 ...
 [ 0.36440909 -0.63622697]
 [ 0.18568733  0.06229272]
 [ 0.48561562 -0.53284399]]<jupyter_text>An Artificial Neural Network (ANN) is an information processing paradigm that is inspired thebrain. ANNs, like people, learn by example. An ANN is configured for a specific application, suchas pattern recognition or data classification, through a learning process. Learning largely involvesadjustments to the synaptic connections that exist between the neurons.The brain consists of hundreds of billion of cells called neurons. These neurons are connectedtogether by synapses which are nothing but the connections across which a neuron can send animpulse to another neuron. When a neuron sends an excitatory signal to another neuron, thenthis signal will be added to all of the other inputs of that neuron. If it exceeds a given thresholdthen it will cause the target neuron to fire an action signal forward — this is how the thinkingprocess works internally.In Computer Science, we model this process by creating “networks” on a computer usingmatrices. These networks can be understood as abstraction of neurons without all the biologicalcomplexities taken into account. To keep things simple, we will just model a simple NN, with twolayers capable of solving linear classification problem.<jupyter_code>model= Sequential()
model.add(Dense(32, input_dim= 2, activation= 'relu'))
model.add(Dense(16, activation= 'relu'))
model.add(Dense(8, activation= 'relu'))
model.add(Dense(4, activation= 'relu'))

model.add(Dense(1, activation= 'sigmoid'))
model.compile(loss='binary_crossentropy', optimizer= 'adam', metrics=['accuracy'])
model.fit(X, y, epochs=200, batch_size= 30)
_, accuracy= model.evaluate(X,y)
print('Accuracy : %.2f' %(accuracy*100))
predict= model.predict(x_test)
rounded= [round(x[0]) for x in predict]
predict= model.predict_classes(x_test)
for i in range(5):
    print('%s => %d(expected %d)' % (x_test[i].tolist(), predict[i], y_test[i]))<jupyter_output>[1.1483513722980445, 0.5695759008186382] => 0(expected 0)
[0.8893510547338092, 0.5652623024498359] => 0(expected 0)
[0.2604082623197106, -0.1781657410236054] => 1(expected 1)
[0.4608354302450183, 0.9109156357718847] => 0(expected 0)
[-0.5938153320331515, 0.47675133662497576] => 0(expected 0)<jupyter_text>ANN using Diabetes Dataset<jupyter_code>diabetes = pd.read_csv('diabetes.csv')
print(diabetes.shape)
diabetes.head(5)
X = diabetes.iloc[:, :-1].values
y = diabetes["Outcome"].values
print("Shape of X Data :",X.shape)
print("Shape of Y Data :",y.shape)
X_train, X_test, Y_train, Y_test = train_test_split(X, y, test_size=0.25, random_state=200)
print("Shape of X Train Data :",X_train.shape)
print("Shape of Y Train Data :",Y_train.shape)
print("Shape of X Test Data :",X_test.shape)
print("Shape of Y Test Data :",Y_test.shape)
normalizer = StandardScaler()
X_train_normalized = normalizer.fit_transform(X_train)
X_test_normalized = normalizer.transform(X_test)
print("Shape of X Train Data :",X_train_normalized.shape)
print("Shape of Y Train Data :",Y_train.shape)
print("Shape of X Test Data :",X_test_normalized.shape)
print("Shape of Y Test Data :",Y_test.shape)
X_test_normalized
model1 = Sequential()
model1.add(Dense(64, input_dim= 8, activation= 'relu'))
model1.add(Dense(32, activation= 'relu'))
model1.add(Dense(16, activation= 'relu'))
model1.add(Dense(8, activation= 'relu'))
model1.add(Dense(1, activation= 'sigmoid'))
model1.summary()
model1.compile(loss="binary_crossentropy",optimizer= 'sgd', metrics=["accuracy"])

history = model1.fit(X_train_normalized, Y_train, epochs=200, batch_size= 30)
hist = pd.DataFrame(history.history)
hist['epoch'] = history.epoch
hist.tail()
history.history.keys()
fig, ax = plt.subplots()
ax.plot(history.history["loss"],'g', marker='.', label="Train Loss")
ax.legend()
_, accuracy= model1.evaluate(X_test_normalized,Y_test)
print('Accuracy : %.2f' %(accuracy*100))
_, accuracy= model1.evaluate(X_train_normalized,Y_train)
print('Accuracy : %.2f' %(accuracy*100))
print('accuracy is {:.3f}'.format(accuracy(y_test,y_pred_class_nn_1)))
print('roc-auc is {:.3f}'.format(roc_auc_score(y_test,y_pred_prob_nn_1)))

plot_roc(y_test, y_pred_prob_nn_1, 'NN')<jupyter_output><empty_output><jupyter_text>Training of model ANN<jupyter_code># initilize parameter
no_example= len(X)
nn_input_dim= 2
nn_output_dim= 2
epsilon= 0.01
reg_lambda=0.01
# loss function calculating
def calculate_loss(model):
    W1, b1,<jupyter_output><empty_output><jupyter_text>Implementation of Artificial Neural Network for AND Logic Gate with 2-bit Binary Input Artificial Neural Network (ANN) is a computational model based on the biological neuralnetworks of animal brains. ANN is modeled with three types of layers: an input layer, hiddenlayers (one or more), and an output layer. Each layer comprises nodes (like biological neurons)are called Artificial Neurons. All nodes are connected with weighted edges (like synapses inbiological brains) between two layers. Initially, with the forward propagation function, theoutput is predicted. Then through backpropagation, the weight and bias to the nodes areupdated to minimizing the error in prediction to attain the convergence of cost function indetermining the final output. ![AND%20gate.png](attachment:AND%20gate.png)  Steps ![approach.png](attachment:approach.png)<jupyter_code>#sigmoid Function
def sigmoid(z):
    return 1/(1+np.exp(-z))

def initlizeParameters(inputFeatures, neuronsInHiddenLayers, outputFeatures):
    W1= np.random.randn(neuronsInHiddenLayers, inputFeatures)
    W2= np.random.randn(outputFeatures, neuronsInHiddenLayers)
    b1= np.zeros((neuronsInHiddenLayers, 1))
    b2= np.zeros((outputFeatures, 1))
    
    parameters= {"W1":W1, "b1":b1, "W2":W2, "b2":b2}
    return parameters

#forward Propagation
def forwardPropagation(X,Y, parameters):
    m= X.shape[1]
    W1= parameters["W1"]
    W2= parameters["W2"]
    b1= parameters["b1"]
    b2= parameters["b2"]
    
    Z1= np.dot(W1, X) + b1
    A1= sigmoid(Z1)
    Z2= np.dot(W2, A1) + b2
    A2= sigmoid(Z2)
    
    cache= (Z1, A1, W1, b1, Z2, A2, W2, b2)
    logprobs= np.multiply(np.log(A2), Y) + np.multiply(np.log(1-A2), (1-Y))
    cost= -np.sum(logprobs) / m
    return cost, cache, A2
#backpropagation
def backwardPropagation(X, Y, cache):
    m= X.shape[1]
    (Z1, A1, W1, b1, Z2, A2, W2, b2)= cache
    
    dZ2= A2-Y
    dW2= np.dot(dZ2, A1.T) / m
    db2= np.sum(dZ2, axis= 1, keepdims=True)
    
    dA1= np.dot(W2.T, dZ2)
    dZ1= np.multiply(dA1, A1 * (1-A1))
    dW1= np.dot(dZ1, X.T) / m
    db1= np.sum(dZ1, axis=1, keepdims=True)/m
    gradients= {"dZ2":dZ2, "dW2":dW2, "db2":db2,
                "dZ1":dZ1, "dW1":dW1,"db1":db1}
    return gradients

#updating the weights based on the negative gadients
def updateParameters(parameters, gradients, learningRate):
    parameters["W1"]= parameters["W1"]- learningRate * gradients["dW1"]
    parameters["W2"]= parameters["W2"]- learningRate * gradients["dW2"]    
    parameters["b1"]= parameters["b1"]- learningRate * gradients["db1"]    
    parameters["b2"]= parameters["b2"]- learningRate * gradients["db2"]    
    return parameters
#model to learn the AND truth table
X= np.array([[0,0,1,1],
            [0,1,0,1]])#AND input
Y= np.array([[0,0,1,1]])#AND output
#define model paramets
neuronsInHiddenLayers= 2 #number of hiddern layers neurons 2
inputFeature= X.shape[0] #number of input features 2
outputFeatures= Y.shape[0] #number of out features 1
parameters= initlizeParameters(inputFeature, neuronsInHiddenLayers, outputFeatures)
epoch= 100000
learning_Rate= 0.01
losses= np.zeros((epoch, 1))

for i in range(epoch):
    losses[i,0], cache, A2= forwardPropagation(X, Y, parameters)
    gradients= backwardPropagation(X,Y, cache)
    parameters= updateParameters(parameters, gradients, learning_Rate)

#evaluating the performance
plt.figure(0)
plt.plot(losses)
plt.xlabel("Epochs")
plt.ylabel("Loss Value")

plt.show()
#testing on new Input data
X= np.array([[1,1,0,0],
            [0,1,0,1]]) #AND Input
cost, _, A2= forwardPropagation(X,Y, parameters)
prediction= (A2>0.5)* 1.0
print(prediction)<jupyter_output>[[1. 1. 0. 0.]]<jupyter_text>Implementation of Artificial Neural Network for OR Logic Gate with 2-bit Binary Input Artificial Neural Network (ANN) is a computational model based on the biological neuralnetworks of animal brains. ANN is modeled with three types of layers: an input layer, hiddenlayers (one or more), and an output layer. Each layer comprises nodes (like biological neurons)are called Artificial Neurons. All nodes are connected with weighted edges (like synapses inbiological brains) between two layers. Initially, with the forward propagation function, theoutput is predicted. Then through backpropagation, the weight and bias to the nodes areupdated to minimizing the error in prediction to attain the convergence of cost function indetermining the final output. ![OR.png](attachment:OR.png)<jupyter_code>#sigmoid Function
def sigmoid(z):
    return 1/(1+np.exp(-z))
def initlizeParameters(inputFeatures, neuronsInHiddenLayers, outputFeatures):
    W1= np.random.randn(neuronsInHiddenLayers, inputFeatures)
    W2= np.random.randn(outputFeatures, neuronsInHiddenLayers)
    b1= np.zeros((neuronsInHiddenLayers, 1))
    b2= np.zeros((outputFeatures, 1))
    
    parameters= {"W1":W1, "b1":b1, 
                 "W2":W2, "b2":b2}
    return parameters

#forward Propagation
def forwardPropagation(X,Y, parameters):
    m= X.shape[1]
    W1= parameters["W1"]
    W2= parameters["W2"]
    b1= parameters["b1"]
    b2= parameters["b2"]
    
    Z1= np.dot(W1, X) + b1
    A1= sigmoid(Z1)
    Z2= np.dot(W2, A1) + b2
    A2= sigmoid(Z2)
    
    cache= (Z1, A1, W1, b1, Z2, A2, W2, b2)
    logprobs= np.multiply(np.log(A2), Y) + np.multiply(np.log(1-A2), (1-Y))
    cost= -np.sum(logprobs) / m
    return cost, cache, A2
def backwardPropagation(X,Y, cache):
    m= X.shape[1]
    (Z1, A1, W1, b1, Z2,A2,W2,b2)= cache
    
    dZ2= A2-Y
    dW2= np.dot(dZ2, A1.T)/ m
    db2= np.sum(dZ2, axis=1, keepdims=True)
    
    dA1= np.dot(W2.T, dZ2)
    dZ1= np.multiply(dA1, A1* (1-A1))
    dW1= np.dot(dZ1, X.T) / m
    db1= np.sum(dZ1, axis=1, keepdims=True)/ m
    
    gradients= {"dZ2":dZ2, "dW2":dW2, "db2":db2,
                "dZ1":dZ1, "dW1":dW1,"db1":db1}
    return gradients

#updating the weights based on the negative gadients
def updateParameters(parameters, gradients, learningRate):
    parameters["W1"]= parameters["W1"]- learningRate * gradients["dW1"]
    parameters["W2"]= parameters["W2"]- learningRate * gradients["dW2"]    
    parameters["b1"]= parameters["b1"]- learningRate * gradients["db1"]    
    parameters["b2"]= parameters["b2"]- learningRate * gradients["db2"]    
    return parameters
#model to learn the OR truth table
X= np.array([[0,0,1,1],
            [0,1,0,1]])#OR input
Y= np.array([[0,1,1,1]])#OR Output
# define model parameters
neuronsInHiddenLayers= 2 #number of hiddern layers neurons 2
inputFeature= X.shape[0] #number of input features 2
outputFeatures= Y.shape[0] #number of out features 1
parameters= initlizeParameters(inputFeature, neuronsInHiddenLayers, outputFeatures)
epoch= 100000
learning_Rate= 0.01
losses= np.zeros((epoch, 1))

for i in range(epoch):
    losses[i,0], cache, A2= forwardPropagation(X, Y, parameters)
    gradients= backwardPropagation(X,Y, cache)
    parameters= updateParameters(parameters, gradients, learning_Rate)
for i in range(epoch):
    losses[i,0], cache, A2= forwardPropagation(X, Y, parameters)
    gradients= backwardPropagation(X,Y, cache)
    parameters= updateParameters(parameters, gradients, learning_Rate)
#evaluating the performance
plt.figure(0)
plt.plot(losses)
plt.xlabel("Epochs")
plt.ylabel("Loss Value")

plt.show()
#testing on new Input data
X= np.array([[1,1,0,0],
            [0,1,0,1]]) #OR Input
cost, _, A2= forwardPropagation(X,Y, parameters)
prediction= (A2>0.5)* 1.0
print(prediction)<jupyter_output>[[1. 1. 0. 1.]]<jupyter_text>Implementation of Artificial Neural Network for NAND Logic Gate with 2-bit Binary Input Artificial Neural Network (ANN) is a computational model based on the biological neuralnetworks of animal brains. ANN is modeled with three types of layers: an input layer, hiddenlayers (one or more), and an output layer. Each layer comprises nodes (like biological neurons) arecalled Artificial Neurons. All nodes are connected with weighted edges (like synapses in biologicalbrains) between two layers. Initially, with the forward propagation function, the output ispredicted. Then through backpropagation, the weight and bias to the nodes are updated tominimizing the error in prediction to attain the convergence of cost function in determining thefinal output. ![NAND%20Gage.png](attachment:NAND%20Gage.png)<jupyter_code>#sigmoid Function
def sigmoid(z):
    return 1/(1+np.exp(-z))
def initlizeParameters(inputFeatures, neuronsInHiddenLayers, outputFeatures):
    W1= np.random.randn(neuronsInHiddenLayers, inputFeatures)
    W2= np.random.randn(outputFeatures, neuronsInHiddenLayers)
    b1= np.zeros((neuronsInHiddenLayers, 1))
    b2= np.zeros((outputFeatures, 1))
    
    parameters= {"W1":W1, "b1":b1, 
                 "W2":W2, "b2":b2}
    return parameters

#forward Propagation
def forwardPropagation(X,Y, parameters):
    m= X.shape[1]
    W1= parameters["W1"]
    W2= parameters["W2"]
    b1= parameters["b1"]
    b2= parameters["b2"]
    
    Z1= np.dot(W1, X) + b1
    A1= sigmoid(Z1)
    Z2= np.dot(W2, A1) + b2
    A2= sigmoid(Z2)
    
    cache= (Z1, A1, W1, b1, Z2, A2, W2, b2)
    logprobs= np.multiply(np.log(A2), Y) + np.multiply(np.log(1-A2), (1-Y))
    cost= -np.sum(logprobs) / m
    return cost, cache, A2

def backwardPropagation(X,Y, cache):
    m= X.shape[1]
    (Z1, A1, W1, b1, Z2,A2,W2,b2)= cache
    
    dZ2= A2-Y
    dW2= np.dot(dZ2, A1.T)/ m
    db2= np.sum(dZ2, axis=1, keepdims=True)
    
    dA1= np.dot(W2.T, dZ2)
    dZ1= np.multiply(dA1, A1* (1-A1))
    dW1= np.dot(dZ1, X.T) / m
    db1= np.sum(dZ1, axis=1, keepdims=True)/ m
    
    gradients= {"dZ2":dZ2, "dW2":dW2, "db2":db2,
                "dZ1":dZ1, "dW1":dW1,"db1":db1}
    return gradients

#updating the weights based on the negative gadients
def updateParameters(parameters, gradients, learningRate):
    parameters["W1"]= parameters["W1"]- learningRate * gradients["dW1"]
    parameters["W2"]= parameters["W2"]- learningRate * gradients["dW2"]    
    parameters["b1"]= parameters["b1"]- learningRate * gradients["db1"]    
    parameters["b2"]= parameters["b2"]- learningRate * gradients["db2"]    
    return parameters

#model to learn the OR truth table
X= np.array([[0,0,1,1],
            [0,1,0,1]])#OR input
Y= np.array([[1,1,1,0]])#OR Output
# define model parameters
neuronsInHiddenLayers= 2 #number of hiddern layers neurons 2
inputFeature= X.shape[0] #number of input features 2
outputFeatures= Y.shape[0] #number of out features 1
parameters= initlizeParameters(inputFeature, neuronsInHiddenLayers, outputFeatures)
epoch= 100000
learning_Rate= 0.01
losses= np.zeros((epoch, 1))

for i in range(epoch):
    losses[i,0], cache, A2= forwardPropagation(X, Y, parameters)
    gradients= backwardPropagation(X,Y, cache)
    parameters= updateParameters(parameters, gradients, learning_Rate)
    
    

    
for i in range(epoch):
    losses[i,0], cache, A2= forwardPropagation(X, Y, parameters)
    gradients= backwardPropagation(X,Y, cache)
    parameters= updateParameters(parameters, gradients, learning_Rate)
#evaluating the performance
plt.figure(0)
plt.plot(losses)
plt.xlabel("Epochs")
plt.ylabel("Loss Value")

plt.show()
#testing on new Input data
X= np.array([[1,1,0,0],
            [0,1,0,1]]) #OR Input
cost, _, A2= forwardPropagation(X,Y, parameters)
prediction= (A2>0.5)* 1.0
print(prediction)
print(A2)<jupyter_output>[[1. 0. 1. 1.]]
[[0.99827464 0.00306389 0.99986965 0.99827552]]
{'fraction_non_alphanumeric': 0.12688925923797484, 'fraction_numerical': 0.051203953795758865, 'mean_word_length': 4.837638376383764, 'pattern_counts': {'":': 30, '<': 24, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '321632', 'license': 'Apache-2.0', 'n_tokens_mistral': 6531, 'n_tokens_neox': 6095, 'n_words': 1753, 'path': 'machine_learning/18_ANN_(using Keras).ipynb', 'repo_name': 'Aliktk/Python_Chilla', 'size': '17401'}