references http neuralnetworksanddeeplearning comchap2 html backpropagation https en wikipedia orgwikisigmoidfunction sigmoid activation function https en wikipedia orgwikifeedforwardneuralnetwork feedforward this function initializes the twohiddenlayerneuralnetwork class with random weights for every layer and initializes predicted output with zeroes inputarray input values for training the neural network i e training data outputarray expected output values of the given inputs input values provided for training the model random initial weights are assigned where first argument is the number of nodes in previous layer and second argument is the number of nodes in the next layer random initial weights are assigned self inputarray shape1 is used to represent number of nodes in input layer first hidden layer consists of 4 nodes random initial values for the first hidden layer first hidden layer has 4 nodes second hidden layer has 3 nodes random initial values for the second hidden layer second hidden layer has 3 nodes output layer has 1 node real output values provided predicted output values by the neural network predictedoutput array initially consists of zeroes the information moves in only one direction i e forward from the input nodes through the two hidden nodes and to the output nodes there are no cycles or loops in the network return layerbetweensecondhiddenlayerandoutput i e the last layer of the neural network inputval numpy array0 0 0 0 0 0 0 0 0 dtypefloat outputval numpy array0 0 0 dtypefloat nn twohiddenlayerneuralnetworkinputval outputval res nn feedforward arraysum numpy sumres numpy isnanarraysum false layerbetweeninputandfirsthiddenlayer is the layer connecting the input nodes with the first hidden layer nodes layerbetweenfirsthiddenlayerandsecondhiddenlayer is the layer connecting the first hidden set of nodes with the second hidden set of nodes layerbetweensecondhiddenlayerandoutput is the layer connecting second hidden layer with the output node function for finetuning the weights of the neural net based on the error rate obtained in the previous epoch i e iteration updation is done using derivative of sogmoid activation function inputval numpy array0 0 0 0 0 0 0 0 0 dtypefloat outputval numpy array0 0 0 dtypefloat nn twohiddenlayerneuralnetworkinputval outputval res nn feedforward nn backpropagation updatedweights nn secondhiddenlayerandoutputlayerweights res updatedweights all false performs the feedforwarding and back propagation process for the given number of iterations every iteration will update the weights of neural network output real output values required for calculating loss iterations number of times the weights are to be updated giveloss boolean value if true then prints loss for each iteration if false then nothing is printed inputval numpy array0 0 0 0 1 0 0 0 1 dtypefloat outputval numpy array0 1 1 dtypefloat nn twohiddenlayerneuralnetworkinputval outputval firstiterationweights nn feedforward nn backpropagation updatedweights nn secondhiddenlayerandoutputlayerweights firstiterationweights updatedweights all false predict s the output for the given input values using the trained neural network the output value given by the model ranges inbetween 0 and 1 the predict function returns 1 if the model value is greater than the threshold value else returns 0 as the real output values are in binary inputval numpy array0 0 0 0 1 0 0 0 1 dtypefloat outputval numpy array0 1 1 dtypefloat nn twohiddenlayerneuralnetworkinputval outputval nn trainoutputval 1000 false nn predict0 1 0 in 0 1 true input values for which the predictions are to be made applies sigmoid activation function return normalized values sigmoidnumpy array1 0 2 1 0 0 dtypenumpy float64 array0 73105858 0 5 0 88079708 0 73105858 0 5 0 5 provides the derivative value of the sigmoid function returns derivative of the sigmoid value sigmoidderivativenumpy array1 0 2 1 0 0 dtypenumpy float64 array 0 0 2 0 0 0 example for how to use the neural network class and use the respected methods for the desired output calls the twohiddenlayerneuralnetwork class and provides the fixed input output values to the model model is trained for a fixed amount of iterations then the predict method is called in this example the output is divided into 2 classes i e binary classification the two classes are represented by 0 and 1 example in 0 1 true input values true output values for the given input values calling neural network class calling training function set giveloss to true if you want to see loss in every iteration this function initializes the twohiddenlayerneuralnetwork class with random weights for every layer and initializes predicted output with zeroes input_array input values for training the neural network i e training data output_array expected output values of the given inputs input values provided for training the model random initial weights are assigned where first argument is the number of nodes in previous layer and second argument is the number of nodes in the next layer random initial weights are assigned self input_array shape 1 is used to represent number of nodes in input layer first hidden layer consists of 4 nodes random initial values for the first hidden layer first hidden layer has 4 nodes second hidden layer has 3 nodes random initial values for the second hidden layer second hidden layer has 3 nodes output layer has 1 node real output values provided predicted output values by the neural network predicted_output array initially consists of zeroes the information moves in only one direction i e forward from the input nodes through the two hidden nodes and to the output nodes there are no cycles or loops in the network return layer_between_second_hidden_layer_and_output i e the last layer of the neural network input_val numpy array 0 0 0 0 0 0 0 0 0 dtype float output_val numpy array 0 0 0 dtype float nn twohiddenlayerneuralnetwork input_val output_val res nn feedforward array_sum numpy sum res numpy isnan array_sum false layer_between_input_and_first_hidden_layer is the layer connecting the input nodes with the first hidden layer nodes layer_between_first_hidden_layer_and_second_hidden_layer is the layer connecting the first hidden set of nodes with the second hidden set of nodes layer_between_second_hidden_layer_and_output is the layer connecting second hidden layer with the output node function for fine tuning the weights of the neural net based on the error rate obtained in the previous epoch i e iteration updation is done using derivative of sogmoid activation function input_val numpy array 0 0 0 0 0 0 0 0 0 dtype float output_val numpy array 0 0 0 dtype float nn twohiddenlayerneuralnetwork input_val output_val res nn feedforward nn back_propagation updated_weights nn second_hidden_layer_and_output_layer_weights res updated_weights all false performs the feedforwarding and back propagation process for the given number of iterations every iteration will update the weights of neural network output real output values required for calculating loss iterations number of times the weights are to be updated give_loss boolean value if true then prints loss for each iteration if false then nothing is printed input_val numpy array 0 0 0 0 1 0 0 0 1 dtype float output_val numpy array 0 1 1 dtype float nn twohiddenlayerneuralnetwork input_val output_val first_iteration_weights nn feedforward nn back_propagation updated_weights nn second_hidden_layer_and_output_layer_weights first_iteration_weights updated_weights all false predict s the output for the given input values using the trained neural network the output value given by the model ranges in between 0 and 1 the predict function returns 1 if the model value is greater than the threshold value else returns 0 as the real output values are in binary input_val numpy array 0 0 0 0 1 0 0 0 1 dtype float output_val numpy array 0 1 1 dtype float nn twohiddenlayerneuralnetwork input_val output_val nn train output_val 1000 false nn predict 0 1 0 in 0 1 true input values for which the predictions are to be made applies sigmoid activation function return normalized values sigmoid numpy array 1 0 2 1 0 0 dtype numpy float64 array 0 73105858 0 5 0 88079708 0 73105858 0 5 0 5 provides the derivative value of the sigmoid function returns derivative of the sigmoid value sigmoid_derivative numpy array 1 0 2 1 0 0 dtype numpy float64 array 0 0 2 0 0 0 example for how to use the neural network class and use the respected methods for the desired output calls the twohiddenlayerneuralnetwork class and provides the fixed input output values to the model model is trained for a fixed amount of iterations then the predict method is called in this example the output is divided into 2 classes i e binary classification the two classes are represented by 0 and 1 example in 0 1 true input values true output values for the given input values calling neural network class calling training function set give_loss to true if you want to see loss in every iteration
import numpy


class TwoHiddenLayerNeuralNetwork:
    def __init__(self, input_array: numpy.ndarray, output_array: numpy.ndarray) -> None:
        

        
        self.input_array = input_array

        
        
        

        
        
        
        self.input_layer_and_first_hidden_layer_weights = numpy.random.rand(
            self.input_array.shape[1], 4
        )

        
        
        
        self.first_hidden_layer_and_second_hidden_layer_weights = numpy.random.rand(
            4, 3
        )

        
        
        
        self.second_hidden_layer_and_output_layer_weights = numpy.random.rand(3, 1)

        
        self.output_array = output_array

        
        
        self.predicted_output = numpy.zeros(output_array.shape)

    def feedforward(self) -> numpy.ndarray:
        
        
        
        self.layer_between_input_and_first_hidden_layer = sigmoid(
            numpy.dot(self.input_array, self.input_layer_and_first_hidden_layer_weights)
        )

        
        
        self.layer_between_first_hidden_layer_and_second_hidden_layer = sigmoid(
            numpy.dot(
                self.layer_between_input_and_first_hidden_layer,
                self.first_hidden_layer_and_second_hidden_layer_weights,
            )
        )

        
        
        self.layer_between_second_hidden_layer_and_output = sigmoid(
            numpy.dot(
                self.layer_between_first_hidden_layer_and_second_hidden_layer,
                self.second_hidden_layer_and_output_layer_weights,
            )
        )

        return self.layer_between_second_hidden_layer_and_output

    def back_propagation(self) -> None:
        

        updated_second_hidden_layer_and_output_layer_weights = numpy.dot(
            self.layer_between_first_hidden_layer_and_second_hidden_layer.T,
            2
            * (self.output_array - self.predicted_output)
            * sigmoid_derivative(self.predicted_output),
        )
        updated_first_hidden_layer_and_second_hidden_layer_weights = numpy.dot(
            self.layer_between_input_and_first_hidden_layer.T,
            numpy.dot(
                2
                * (self.output_array - self.predicted_output)
                * sigmoid_derivative(self.predicted_output),
                self.second_hidden_layer_and_output_layer_weights.T,
            )
            * sigmoid_derivative(
                self.layer_between_first_hidden_layer_and_second_hidden_layer
            ),
        )
        updated_input_layer_and_first_hidden_layer_weights = numpy.dot(
            self.input_array.T,
            numpy.dot(
                numpy.dot(
                    2
                    * (self.output_array - self.predicted_output)
                    * sigmoid_derivative(self.predicted_output),
                    self.second_hidden_layer_and_output_layer_weights.T,
                )
                * sigmoid_derivative(
                    self.layer_between_first_hidden_layer_and_second_hidden_layer
                ),
                self.first_hidden_layer_and_second_hidden_layer_weights.T,
            )
            * sigmoid_derivative(self.layer_between_input_and_first_hidden_layer),
        )

        self.input_layer_and_first_hidden_layer_weights += (
            updated_input_layer_and_first_hidden_layer_weights
        )
        self.first_hidden_layer_and_second_hidden_layer_weights += (
            updated_first_hidden_layer_and_second_hidden_layer_weights
        )
        self.second_hidden_layer_and_output_layer_weights += (
            updated_second_hidden_layer_and_output_layer_weights
        )

    def train(self, output: numpy.ndarray, iterations: int, give_loss: bool) -> None:
        
        for iteration in range(1, iterations + 1):
            self.output = self.feedforward()
            self.back_propagation()
            if give_loss:
                loss = numpy.mean(numpy.square(output - self.feedforward()))
                print(f"Iteration {iteration} Loss: {loss}")

    def predict(self, input_arr: numpy.ndarray) -> int:
        

        
        self.array = input_arr

        self.layer_between_input_and_first_hidden_layer = sigmoid(
            numpy.dot(self.array, self.input_layer_and_first_hidden_layer_weights)
        )

        self.layer_between_first_hidden_layer_and_second_hidden_layer = sigmoid(
            numpy.dot(
                self.layer_between_input_and_first_hidden_layer,
                self.first_hidden_layer_and_second_hidden_layer_weights,
            )
        )

        self.layer_between_second_hidden_layer_and_output = sigmoid(
            numpy.dot(
                self.layer_between_first_hidden_layer_and_second_hidden_layer,
                self.second_hidden_layer_and_output_layer_weights,
            )
        )

        return int((self.layer_between_second_hidden_layer_and_output > 0.6)[0])


def sigmoid(value: numpy.ndarray) -> numpy.ndarray:
    
    return 1 / (1 + numpy.exp(-value))


def sigmoid_derivative(value: numpy.ndarray) -> numpy.ndarray:
    
    return (value) * (1 - (value))


def example() -> int:
    
    
    test_input = numpy.array(
        (
            [0, 0, 0],
            [0, 0, 1],
            [0, 1, 0],
            [0, 1, 1],
            [1, 0, 0],
            [1, 0, 1],
            [1, 1, 0],
            [1, 1, 1],
        ),
        dtype=numpy.float64,
    )

    
    output = numpy.array(([0], [1], [1], [0], [1], [0], [0], [1]), dtype=numpy.float64)

    
    neural_network = TwoHiddenLayerNeuralNetwork(
        input_array=test_input, output_array=output
    )

    
    
    neural_network.train(output=output, iterations=10, give_loss=False)

    return neural_network.predict(numpy.array(([1, 1, 1]), dtype=numpy.float64))


if __name__ == "__main__":
    example()