<jupyter_start><jupyter_text>EffortlessAI: Multilayer Perceptron Tool*Version 1.2**Last Updated: 2/6/2018*  *Programmed By:* <NAME>    Supports:  * Loading/saving previous neural networks* Training via gradient descent* Custom amounts of layers and nodes* Visual representation of weights via matplotlib* Custom epoch progression* Training sets/Multiple queries* Viewing/querying previous epochs<jupyter_code>#Imports
import numpy as np
import matplotlib.pyplot as plt
import csv
import json
import copy
import scipy.special
import ipywidgets as widgets
from IPython.display import Markdown, display, clear_output

np.set_printoptions(threshold=np.nan)
#Initializes console and output widgets
def printmd(string):
    display(Markdown(string))

printmd('### ** Console**')
console = widgets.Output(
    readout=True,
    readout_format='d',
    layout=widgets.Layout(height='150px', border='solid 1px')
)
display(console)
    
out = widgets.Output()
with out: #Nerual network initialization tools
    printmd('### **New Neural Network Initialization**')
display(out)

load = widgets.Output()
with load: #Neural network loading tools
    printmd('### **Load a Neural Network**')
display(load)
    
out2 = widgets.Output() #Neural network interaction tools
display(out2)

out3 = widgets.Output() #Subplots for NN weights
display(out3)

out4 = widgets.Output() #Querying
display(out4)

out5 = widgets.Output() #Training
display(out5)

out6 = widgets.Output() #Saving
display(out6)

#Declares neural network as global so that it can be easily accessed
global n<jupyter_output><empty_output><jupyter_text>**-------------------------------------------------** Documentation New Neural Network Initialization* Users are able to quickly initialize their own neural networks through seven widgets.* Number of Hidden Layers Widget:    * Describes the number of layers between the input layer and output layer. As of now, we only support a maximum of 3 hidden layers* Number of Nodes for Input Layer:    * Must match with the size of the inputs used to query and train the neural network* Number of Nodes per Hidden Layer:    * As this number increases, the ability of the neural network to recognize increasingly complicated non-linear patterns does as well.    * As of now, all hidden layers must have the same number of nodes.* Number of Nodes for Output Layer:    * Must match with the size of the targets used to train the neural network* Initial Weight Generation Method:    * Users can have weights randomly generated between -0.5 and +0.5.    * Users can have weights generated via a normal distribution where the mean occurs at zero and the standard deviation is equal to ${\frac {1}{\sqrt{incoming\space links}}}$* Activation Function:    * Users can select between a sigmoid (${\frac {1}{1+e^{-x}}}$) and hyperbolic tangent activation function.* Learning Rate:    * The most popular learning rates used for multi-layer perceptron training are included, ranging from 0.00001 to 10. Loading a Neural Network* In order to load neural network, users must first paste the JSON object data of a previously saved EffortlessAI neural network into the "JSON" text box.* Next, they must place the weights array of a previously saved EffortlessAI neural network into the "weights" text box.   Neural Network Viewing* Users have access to a display that lists current characterics of the neural network and updates dynamically.* Users are able to visualize the weights of previous epochs and subsequent learning progression of a neural network via a slider that generates weight-modeling graphics for each epoch. Querying* Querying refers to the propagation of inputs through the neural network in order to calculate their respective outputs.* Users are able to query multiple sets at a time.* Formatting:    * [[first query], [second query], [third query], [nth query]] where each query is a comma-separated list of numbers        * *Example (for a neural network with 3 input nodes):* [[1, 3, 4], [5, 2, 7], [1, 0.9, 4]]    * Outputs are structured in the same way as query inputs.        * *Example:* [[Output of first query], [Output of second query], [Output of third query]]* Users are able to query either the most recent epoch of the neural network or a previous epoch.    * In order to query the most recent epoch, ensure that "Epoch to Query" is set to "Most Recent Epoch"    * In order to query a previous epoch, set the "Currently Selected Epoch" slider in "Currently Loaded Neural Network" section to your desired epoch. Then, make sure that "Epoch to Query" is set to "Currently Selected Epoch"   Training* Training data is structured identically to queries. However, both inputs and targets must be specified.* Formatting:    * Inputs: [[first set of inputs], [second set of inputs], [third set of inputs], [nth set of inputs]]        * *Example (for a neural network with 3 input nodes):* [[1, 3, 4], [5, 2, 7], [1, 0.9, 4]]    * Targets: [[first set of targets], [second set of targets], [third set of targets], [nth set of targets]]        * *Example (for a neural network with 3 output nodes):* [[0.15, 0.93, 0.24], [0.45, 0.192, 0.37], [0.11, 0.89, 0.114]]* The number of training iterations can be set via an integer slider    * For a large number of training iterations, the screen may appear to temporarily freeze while calculations are being performed. This is normal. Saving* As of now, saving works but is relatively messy and in its fledgling stages.* Neural networks with thousands of training epochs contain an extremely large amount of data.    * Users are able to select the range of epochs that they would like to save. For example, if a user trained their neural network with 10,000 iterations of training data, they may only want to save epochs 9,000 through 10,000; this will give them a file that is nearly 1/10th the size of what it would be.* After setting the desired epoch saving settings, click the "Save" button.* An algorithm will generate a JSON object that stores the characteristics of the neural network. The weights will be outputted in a specially formatted array.* Save both the JSON object and weights array on your local computer. They can be pasted into the "Loading a Neural Network" section later on. Known Bugs* If initializing a new neural network while one is currently loaded, the subsequent weight graphics will have a delayed loading time when switching between epochs. This can be avoided by restrating the kernal whenever dealing with a new neural network.* Occassionally extra weight visualization graphics will appear after initialization. This is relatively uncommon, but if it happens, refresh the page. Upcoming Features* Including a converter that turns CSV files into properly formatted queries/training data* Improved save/load format* Ability to generate animations that demonstrate learning* Ability to create interactive charts that demonstrate performance in relation to test data for selected epochs* Ability to customize the normal generation weight generation method* More options for learning rates* More options for activation functions* Improved console messages* Optimized code; should speed up this software significantly* More interpolations for weight-representing graphics<jupyter_code>#Initializes Neural Network Initialization Widgets

#Number of Hidden Layers Int Slider Widget & Associated Label
LWSlider =  widgets.IntSlider(
        value=1,
        min=1,
        max=3,
        step=1,
        disabled=False,
        continuous_update=False,
        orientation='horizontal',
        readout=True,
        readout_format='d')
LayersWidget = widgets.HBox([
    widgets.Label(
        value="Number of Hidden Layers:",
        layout=widgets.Layout(width='250px')),
    LWSlider
])

#Nodes Per Input Layer Int Slider Widget & Associated Label
NPILSlider = widgets.IntSlider(
    value=200,
        min=3,
        max=1000,
        step=1,
        disabled=False,
        continuous_update=False,
        orientation='horizontal',
        readout=True,
        readout_format='d')
NodesPerInputLayerWidget = widgets.HBox([
    widgets.Label(
        value="Number of Nodes for Input Layer:",
        layout=widgets.Layout(width='250px')),
    NPILSlider
])

#Nodes Per Hidden Layer Int Slider Widget & Associated Label
NPHLSlider = widgets.IntSlider(
    value=200,
        min=3,
        max=1000,
        step=1,
        disabled=False,
        continuous_update=False,
        orientation='horizontal',
        readout=True,
        readout_format='d')
NodesPerHiddenLayerWidget = widgets.HBox([
    widgets.Label(
        value="Number of Nodes per Hidden Layer:",
        layout=widgets.Layout(width='250px')),
    NPHLSlider
])

#Nodes Per Output Layer Int Slider Widget & Associated Label
NPOLSlider = widgets.IntSlider(
    value=200,
        min=3,
        max=1000,
        step=1,
        disabled=False,
        continuous_update=False,
        orientation='horizontal',
        readout=True,
        readout_format='d')
NodesPerOutputLayerWidget = widgets.HBox([
    widgets.Label(
        value="Number of Nodes for Output Layer:",
        layout=widgets.Layout(width='250px')),
    NPOLSlider
])

#Weight Generation Method Radio Button Widget & Associated Label
WGMRadioButton = widgets.RadioButtons(
    options=['Random between -0.5 and +0.5', 'N(μ,σ^2) for σ^2=1/sqrt(incoming links)'],
    value='Random between -0.5 and +0.5',
    disabled=False,)
WeightGenerationMethodWidget = widgets.HBox([
    widgets.Label(
        value="Initial Weight Generation Method:",
        layout=widgets.Layout(width='250px')),
    WGMRadioButton
])

#Activation Function Radio Button Widget & Associated Label
AFRadioButton = widgets.RadioButtons(
    options=['Sigmoid', 'Tanh'],
    value='Sigmoid',
    disabled=False,)
ActivationFunctionWidget = widgets.HBox([
    widgets.Label(
        value="Activation Function:",
        layout=widgets.Layout(width='250px')),
    AFRadioButton
])

#Learning Rate Dropdown Widget
LRDropdown = widgets.Dropdown(
    options=[0.00001, 0.0001, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1.0, 3.0, 10.0],
    value=0.03,
    disabled=False,)
LearningRateWidget = widgets.HBox([
    widgets.Label(
        value="Learning Rate:"),
    LRDropdown
])

#Button to initialize a new neural network
SubmitNNWidget = widgets.Button(
    description='Initialize',
    disabled=False,
    button_style='info',
    tooltip='Initialize Neural Network',
    icon='',
)
#Groups and Displays Neural Network Initialization Widgets
left_box = widgets.VBox([LayersWidget, NodesPerInputLayerWidget, NodesPerHiddenLayerWidget, NodesPerOutputLayerWidget])
right_box = widgets.VBox([WeightGenerationMethodWidget, ActivationFunctionWidget, LearningRateWidget])
outputs = widgets.HBox([left_box, right_box])
outputs2 = widgets.HBox([SubmitNNWidget])

#Prints initialization widgets into the correct output widget
with out:
    display(outputs)
    display(outputs2)
#Rich text widget that will intake the JSON data
LoadInput1 = widgets.Textarea(
    value="",
    placeholder='',
    description='JSON:',
    disabled=False,
    layout = widgets.Layout(width='400px')
)

#Rich text input that will intake the weights data
LoadInput2 = widgets.Textarea(
    value="",
    placeholder='',
    description='Weights:',
    disabled=False,
    layout = widgets.Layout(width='400px')
)

#Button to load the inputted neural network
LoadNNWidget = widgets.Button(
    description='Load NN',
    disabled=False,
    button_style='info',
    tooltip='Load NN',
    icon='',
    layout=widgets.Layout(margin='0px 0px 0px 0px')
)

#Formats and displays widgets
LoadBox = widgets.HBox([LoadInput1, LoadInput2])
LoadBoxLower = widgets.HBox([LoadNNWidget])

with load:
    display(LoadBox, LoadBoxLower)
#Neural Network Class Definition
class neuralNetwork():
    
    #initialization function
    def __init__(self, status, layers, inputnodes, hiddennodes, outputnodes, learningrate, activationfunction, weightinitialization):
        
        #set number of nodes in input, hidden, and output layers
        self.inodes = inputnodes
        self.hnodes = hiddennodes
        self.onodes = outputnodes
        self.epochs = 1
        self.actfunction = activationfunction
        self.methodgen = weightinitialization

        #number of hidden layers
        self.layers = layers

        #learning rate
        self.lr = learningrate

        #activation function
        if(activationfunction == "Sigmoid"):
            self.activation_function = lambda x: scipy.special.expit(x)
        elif(activationfunction == "Tanh"):
            self.activation_function = lambda x: np.tanh(x)

        #weights initialization
        #the weights array is made up of additional arrays representing the weights for each epoch
        #inside each epoch array is another set of arrays, where the indeces (low to high) represents the synpapses (left to right)
        #weights inside those arrays are w_i_j, where link is from node i to node j in the next layer
        #w11 w21
        #w12 w22 etc.
        
        #For a newly initialized neural network
        if(status == "new"):
            
            self.weights = [[]]

            if(weightinitialization=='Random between -0.5 and +0.5'):

                #weights between input and hidden layer
                self.weights[0].append((np.random.rand(self.hnodes, self.inodes) - 0.5)) 
                #rows = number of hidden nodes; columns = number of input nodes

                #adds hidden layer weights
                for x in range(layers-1):
                    self.weights[0].append((np.random.rand(self.hnodes, self.hnodes) - 0.5))

                #weights between hidden and output layer
                self.weights[0].append((np.random.rand(self.onodes, self.hnodes) - 0.5))     
            
            if(weightinitialization=='N(μ,σ^2) for σ^2=1/sqrt(incoming links)'): #samples a normal distribution where the mean is zero and the st. dev is the 1/sqrt(incoming links)
                #weights between input and hidden layer
                self.weights[0].append((np.random.normal(0.0, pow(self.hnodes, -0.5), (self.hnodes, self.inodes)))) 

                #adds hidden layer weights
                for x in range(layers-1):
                    self.weights[0].append((np.random.normal(0.0, pow(self.hnodes, -0.5), (self.hnodes, self.hnodes))))

                #weights between hidden and output layer
                self.weights[0].append((np.random.normal(0.0, pow(self.onodes, -0.5), (self.onodes, self.hnodes))))
                print(self.weights)
            
            #initializes widgets for viewing and interaction
            createInteract(self)
        
        #Called if a neural network is being reloaded rather than initialized from scratch
        elif(status=="load"):
            
            #Sets temporary variable equal to weights
            temp = str(LoadInput2.value)
            
            #Cleans up string
            temp = temp.replace('[', '')
            temp = temp.replace(']', '')
            temp = temp.replace('array', '')
            temp = temp.replace('(', '')
            temp = temp.replace('', '')
            templist = temp.split(',')
            
            #will be used later on
            templist2 = []
            
            #makes it easier to divide the list into epochs
            sizeofeachepoch = self.inodes*self.hnodes + self.hnodes*self.hnodes*(self.layers-1) + self.hnodes*self.onodes
            
            #adds epoch dimension
            for x in range(0, len(templist)):
                #adds dimensions for each epoch
                if (x%(sizeofeachepoch)==0 and x != 1):
                    templist2.append(templist[x:x+sizeofeachepoch])
            
            #breaks up each epoch dimension into 2D numpy arrays representing the weights between each layer
            #This function is extremely messy and ineffecient; it will be optimized in the near future
            for x in range(0, len(templist2)):
                sublist = []
                if layers==1:
                    sublist.append((np.array(templist2[x][0:self.inodes*self.hnodes]).reshape(self.hnodes, self.inodes)).astype('float_'))
                    sublist.append((np.array(templist2[x][self.inodes*self.hnodes:self.inodes*self.hnodes+self.hnodes*self.onodes]).reshape(self.onodes, self.hnodes)).astype('float_'))
                if layers==2:
                    sublist.append((np.array(templist2[x][0:self.inodes*self.hnodes]).reshape(self.hnodes, self.inodes)).astype('float_'))
                    sublist.append((np.array(templist2[x][self.inodes*self.hnodes:self.hnodes*self.hnodes+self.inodes*self.hnodes]).reshape(self.hnodes, self.hnodes)).astype('float_'))
                    sublist.append((np.array(templist2[x][self.hnodes*self.hnodes+self.inodes*self.hnodes:self.hnodes*self.hnodes+self.inodes*self.hnodes+self.hnodes*self.onodes]).reshape(self.onodes, self.hnodes)).astype('float_'))
                elif layers==3:
                    sublist.append((np.array(templist2[x][0:self.hnodes*self.hnodes]).reshape(self.hnodes, self.hnodes)).astype('float_'))
                    sublist.append((np.array(templist2[x][self.inodes*self.hnodes:self.hnodes*self.hnodes+self.inodes*self.hnodes]).reshape(self.hnodes, self.hnodes)).astype('float_'))
                    sublist.append((np.array(templist2[x][self.inodes*self.hnodes + self.hnodes*self.hnodes+self.inodes*self.hnodes:self.hnodes*self.onodes+self.inodes*self.hnodes + self.hnodes*self.hnodes+self.inodes*self.hnodes]).reshape(self.onodes, self.hnodes)).astype('float_'))
                    sublist.append((np.array(templist2[x][self.hnodes*self.onodes+self.inodes*self.hnodes + self.hnodes*self.hnodes+self.inodes*self.hnodes:self.hnodes*self.onodes+self.inodes*self.hnodes + self.hnodes*self.hnodes+self.inodes*self.hnodes+self.hnodes*self.onodes]).reshape(self.onodes, self.hnodes)).astype('float_'))

                templist2[x] = sublist
                
            self.weights = templist2
            
            #sets epoch variable
            self.epochs = len(self.weights)
            
            #updates viewing widgets
            EpochSlider.max = self.epochs
            EpochSaves.max = self.epochs
            Epochs.value = 'Number of Epochs (1 = initialization): ' + str(self.epochs)
        
            #initializes widgets for viewing and interaction 
            createInteract(self)
    
    #Inputs_list should be an array containing arrays representing individual training inputs
    #Note: any comments labeled with 'test' are included so that training can be visually tested if needed; just remove the '#'
    def train(self, inputs_list, targets_list):
        
        #test: print("First query: " + str(self.query([[5, 2, 4]])))
        
        #convert inputs list to 2d array
        inputs = []
        targets = []
        
        for x in range(len(inputs_list)):
            inputs.append(np.array(inputs_list[x], ndmin=2).T)
            targets.append(np.array(targets_list[x], ndmin=2).T)
        
        #test: print("Inputs: " + str(inputs))
        #test: print("Targets: " + str(targets))
        
        #copy function prevents changes to newweightsarray from affecting the actual weights of the previous epoch
        newweightsarray = copy.copy(self.weights[self.epochs-1])
        
        #test: print("New Weights Array: " + str(newweightsarray))
        
        #Iterates through for the entire training set and then only increases the epoch once.
        for z in range(len(inputs)):
            inputstorage = []
            outputstorage = []
            errorstorage = []

            #creates an array containing the inputs into each layer of the NN with the given training inputs
            #index 0 => input into the first hidden layer
            #highest index => input into the output layer
            for x in range(len(self.weights[0])):
                if x==0:
                    inputstorage.append(np.dot(newweightsarray[x], inputs[z]))
                else:
                    inputstorage.append(np.dot(newweightsarray[x], self.activation_function(inputstorage[x-1])))
            
            #test: print("Inputs Storage:" + str(inputstorage))
            
            #creates an array containing the output from each layer of the NN with the given training inputs
            #index 0 => input from the first hidden layer
            #highest index => input into the output layer
            for x in range(len(self.weights[0])):
                outputstorage.append(self.activation_function(inputstorage[x]))
            
            #test:  print("Output Storage:" + str(outputstorage))
            
            #creates an array the holds the errors for each layer
            #index 0 => errors for the first hidden layer
            #highest index => errors for the output layer
            #the error for layer n is: the dot product of the transposed weights between layer n and n+1 and the error of layer n+1

            #output errors are just targets - outputs of the output array, so this is done outside of the loop to avoid executing
            #repretitive boolean functions for no reason
            errorstorage.append(targets[z] - outputstorage[len(outputstorage)-1])

            for x in range(len(self.weights[0])-1, 0, -1):
                errorstorage.insert(0, np.dot(newweightsarray[x].T, errorstorage[0]))
            
            #test: print("Error Storage:" + str(errorstorage))
            
            change = []
            
            #finds the new weights by using gradient descent
            for x in range(len(newweightsarray)):
                if x==0: #between input and hidden
                    change.append(self.lr * np.dot((errorstorage[x] * outputstorage[x] * (1.0 - outputstorage[x])), np.transpose(inputs[z])))    
                else: #between hidden and output
                    change.append(self.lr * np.dot((errorstorage[x] * outputstorage[x] * (1.0 - outputstorage[x])), np.transpose(outputstorage[x-1])))
            
            #test: print("Change:" + str(change))
            
            for x in range(len(newweightsarray)):
                change[x] += newweightsarray[x]
                
            newweightsarray = copy.copy(change)
            
            #test: print("New Weights:" + str(newweightsarray))
            
        #increments epoch
        self.epochs = self.epochs + 1
        
        #adds the array to hold the new epoch
        self.weights.append([])

        #fills the previously added array with the weights between each layer
        for x in range(len(newweightsarray)):
            self.weights[self.epochs-1].append(newweightsarray[x])
        
        EpochSlider.max = self.epochs
        EpochSaves.max = self.epochs
        Epochs.value = 'Number of Epochs (1 = initialization): ' + str(self.epochs)
        
    def query(self, inputs_list, epoch):
        
        inputs = []

        #convert inputs_list into 2d array
        for z in range(len(inputs_list)):
            inputs.append(np.array(inputs_list[z], ndmin=2).T)
            #ndmin is a numpy parameter specifying the minimum number of dimensions and array shold have
            #.T transposes the array; turns first column into first row, second column into second row
        
        outputlist = []
        
        for z in range(len(inputs)):
            #storage array for output after query
            outputarray = inputs[z]

            for x in range(0, len(self.weights[epoch-1])):
                if x==0:
                    outputarray = np.dot(self.weights[epoch-1][x], inputs[z])
                    outputarray = self.activation_function(outputarray)
                else:
                    outputarray = np.dot(self.weights[epoch-1][x], outputarray)
                    outputarray = self.activation_function(outputarray)

            outputlist.append(outputarray)
        
        return outputlist
    
    #Converts the characteristics of neural network in JSON format
    def toJSON(self, lower, upper):
        data = {}
        data['inodes'] = self.inodes
        data['hnodes'] = self.hnodes
        data['onodes'] = self.onodes
        data['actfunction'] = self.actfunction
        data['methodgen'] = self.methodgen
        data['layers'] = self.layers
        data['lr'] = self.lr
        return(json.dumps(data))
    
    #Returns the weights of neural network
    def printWeights(self, lower, upper):
        outputdata = []
        for x in range(lower-1, upper):
            outputdata.append(self.weights[x])
        return(outputdata)
    
#Function that is called when the initialization button is clicked
def InitializeNN(b):
    n = neuralNetwork("new", LWSlider.value, NPILSlider.value, NPHLSlider.value, NPOLSlider.value, LRDropdown.value, AFRadioButton.value, WGMRadioButton.value)
    with console:
        print("Neural Network Successfully Initialized:")
        print('\t* Number of Hidden Layers: ' + str(LWSlider.value))
        print('\t* Number of Nodes for Input Layer: ' + str(NPILSlider.value))
        print('\t* Number of Nodes per Hidden Layer: ' + str(NPHLSlider.value))
        print('\t* Number of Nodes for Output Layer: ' + str(NPOLSlider.value))
        print('\t* Learning Rate: ' + str(LRDropdown.value))
        print('\t* Activation Function: ' + AFRadioButton.value)
        print('\t* Method of Initial Weight Generation: ' + WGMRadioButton.value)
    
#Links initialization button to initialization function
SubmitNNWidget.on_click(InitializeNN)

#Function that is called when the loading button is clicked
def LoadNN(b):
    try:
        NNJSON = json.loads(str(LoadInput1.value))
        inodes = NNJSON['inodes']
        hnodes = NNJSON['hnodes']
        onodes = NNJSON['onodes']
        methodgen = NNJSON['methodgen']
        actfunction = NNJSON['actfunction']
        layers = NNJSON['layers']
        lr = NNJSON['lr']
        try:
            n = neuralNetwork("load", layers, inodes, hnodes, onodes, lr, actfunction, methodgen)
        except:
            with console:
                print("Improper weights format.")
    except:
        with console:
            print("Improper JSON format.")

#Links loading button to loading function
LoadNNWidget.on_click(LoadNN)
#Initializes Neural Network Interaction Widgets

#Placed on outside for easy access by other code blocks. Doesn't really matter so long as its display is in the function
EpochSlider = widgets.IntSlider(
        value=1,
        min=1,
        max=1,
        step=1,
        disabled=False,
        continuous_update=False,
        orientation='horizontal',
        readout=True,
        readout_format='d',
        layout=widgets.Layout(width='400px'))

#Number of Trained Epochs. Created out here so that it can be easily accessed by other code blocks (not possible if in function)
Epochs = widgets.Label(value = 'Number of Epochs (1 = initialization): 1')

#Creates and displays widgets used in viewing the current neural network
def createInteract(neuralnetwork):
    #Clears previous output in order to avoid duplicate outputs
    with out2:
        clear_output()
        printmd('### **Currently Loaded Neural Network**')
        
    #Widget for number of hidden layers of current NN
    HiddenLayers = widgets.Label(value = 'Number of Hidden Layers: ' + str(neuralnetwork.layers), layout=widgets.Layout(width='400px'))
    
    #Widget for number of input nodes of current NN
    InputNodes = widgets.Label(value = 'Number of Input Nodes: ' + str(neuralnetwork.inodes))
    
    #Widget for number of output nodes of current NN
    OutputNodes = widgets.Label(value = 'Number of Output Nodes: ' + str(neuralnetwork.onodes))
    
    #Widget for number of hidden nodes of current NN
    HiddenNodes = widgets.Label(value = 'Number of Nodes per Hidden Layer: ' + str(neuralnetwork.hnodes))
    
    #Widget for learning rate of current NN
    LearningRate = widgets.Label(value = 'Learning Rate: ' + str(neuralnetwork.lr))
    
    #Widget for activation function of current NN
    ActivationFunction = widgets.Label(value = 'Activation Function: ' + str(neuralnetwork.actfunction))
    
    #Widget for method of generation of current NN
    Methodgen = widgets.Label(value = 'Method of Initial Weight Generation: ' + str(neuralnetwork.methodgen))
    
    #groups widgets into a left-side box
    left_box1 = widgets.VBox([HiddenLayers, InputNodes, OutputNodes, HiddenNodes])
    
    #groups widgets into a right-side box
    right_box1 = widgets.VBox([LearningRate, ActivationFunction, Methodgen, Epochs])
    
    #groups left_box1 and right_box1 into a centered box for output
    outputs1 = widgets.HBox([left_box1, right_box1])
    
    #Select epoch to view weights of previous epochs
    EpochSlider.max=neuralnetwork.epochs
    EpochUp = widgets.Button(description = "+", button_style='info')
    EpochDown = widgets.Button(description = "-", button_style='info')
    EpochWidget = widgets.HBox([
        widgets.Label(
            value="Currently Selected Epoch:",
            layout=widgets.Layout(width='200px')),
        EpochSlider,
        EpochDown,
        EpochUp
    ])
    
    with out2:
        display(outputs1)
        display(EpochWidget)
        
    genweightplots(EpochSlider.value, neuralnetwork)
    
    #Remakes the weight images when the slider is changed. Must be in here to have access to local vars easily
    def Sliderchange(slider):
        genweightplots(EpochSlider.value, neuralnetwork)
        
    EpochSlider.observe(Sliderchange, names='value')
    
    def bringEpochUp(b):
        EpochSlider.value = EpochSlider.value + 1
    
    def bringEpochDown(b):
        EpochSlider.value = EpochSlider.value - 1
    
    EpochDown.on_click(bringEpochDown)
    EpochUp.on_click(bringEpochUp)
    
    createQuery(neuralnetwork)
    createTrain(neuralnetwork)
    createSave(neuralnetwork)
#Function for creating the subplots representing weights and then displaying them in out3 
def genweightplots(epochnum, neuralnetwork):
    with out3:
        clear_output()
    f, axs = plt.subplots(2,2,figsize=(15,5))
    weightarraynum = len(neuralnetwork.weights[0])
    
    #Here for testing
    #print(neuralnetwork.weights)
    #print(len(neuralnetwork.weights))
    
    for x in range(0, weightarraynum):
        plt.subplot(1, weightarraynum, x+1)
        plt.imshow(neuralnetwork.weights[epochnum-1][x], interpolation="nearest")
        if x==0:
            plt.title("Layer " + str(x+1) + " (Input) & " + str(x+2))
        elif x==(len(neuralnetwork.weights[epochnum-1])-1):
            plt.title("Layer " + str(x+1) + " & " + str(x+2) + " (Output)")
        else:
            plt.title("Layer " + str(x+1) + " & " + str(x+2))
    with out3:
        plt.show()
#Create query widgets

queryoutput = []

def createQuery(neuralnetwork):
    with out4:
        clear_output()
        printmd('### **Query**')
        
    QueryInput = widgets.Textarea(
        value="",
        placeholder='See documentation for query formatting',
        description='Query Input:',
        disabled=False,
        layout = widgets.Layout(width='400px')
    )
    
    QueryOutput = widgets.Textarea(
        value="",
        description='Output:',
        disabled=False,
        layout = widgets.Layout(width='400px')
    )
    
    EpochtoQuery = widgets.RadioButtons(
        options=['Most Recent Epoch', 'Currently Selected Epoch'],
        value='Most Recent Epoch',
        layout=widgets.Layout(width='300px'),
        disabled=False
    )
    
    EpochtoQueryWidget = widgets.HBox([
        widgets.Label(
            value="Epoch to Query:",
            layout=widgets.Layout(width='200px')),
        EpochtoQuery
    ], layout=widgets.Layout(width='600px'))
    
    QueryButton = widgets.Button(description = "Query", button_style='info')
    
    #Future feature
    #SaveButton = widgets.Button(description = "Save Output (csv)", button_style='info', layout=widgets.Layout(margin='2px 0px 0px 50px'))
    
    Queryboxestop = widgets.HBox([QueryInput, QueryOutput])
    Queryboxesbottom = widgets.HBox([EpochtoQueryWidget, QueryButton])
    
    def QueryFunction(b):
        try:
            listofinput = json.loads(QueryInput.value)
            if len(listofinput[0]) != neuralnetwork.inodes:
                QueryOutput.value = "Incorrect number of input values."
            else:
                if EpochtoQuery.value=='Most Recent Epoch':
                    queryoutput = neuralnetwork.query(listofinput, neuralnetwork.epochs-1)
                elif EpochtoQuery.value=='Currently Selected Epoch':
                    queryoutput = neuralnetwork.query(listofinput, EpochSlider.value)
                QueryOutput.value=str(queryoutput)
        except:
            with console:
                print("Incorrect query format.")

    QueryButton.on_click(QueryFunction)
    
    
    with out4:
        display(Queryboxestop, Queryboxesbottom)
#Creates training widgets

def createTrain(neuralnetwork):
    with out5:
        clear_output()
        printmd('### **Train**')
        
    TargetsInput = widgets.Textarea(
        value="",
        placeholder='See documentation for input list formatting',
        description='Inputs List:',
        disabled=False,
        layout = widgets.Layout(width='400px')
    )
    
    TargetsOutput = widgets.Textarea(
        value="",
        placeholder='See documentation for target list formatting',
        description='Targets List:',
        disabled=False,
        layout = widgets.Layout(width='400px')
    )
    
    TrainingIterations = widgets.IntSlider(
        min=0,
        max=500,
        value=1,
        layout=widgets.Layout(width='300px')
    )
    
    TrainingIterationsWidge = widgets.HBox([
        widgets.Label(value="Training Iterations:", layout=widgets.Layout(width='200px')),
        TrainingIterations    
    ], layout=widgets.Layout(width = '600px'))
    
    TrainButton = widgets.Button(description = "Train", button_style='info')
    
    Trainboxestop = widgets.HBox([TargetsInput, TargetsOutput])
    Trainboxesbottom = widgets.HBox([TrainingIterationsWidge, TrainButton])
    
    def TrainFunction(b):
        try:
            listofinputs = json.loads(TargetsInput.value)
            listoftargets = json.loads(TargetsOutput.value)
            if len(listofinputs[0]) != neuralnetwork.inodes:
                with console:
                    print("Incorrect number of input values.")
            elif len(listoftargets[0]) != neuralnetwork.onodes:
                with console:
                    print("Incorrect number of target values.")
            else:
                for x in range(TrainingIterations.value):
                    neuralnetwork.train(listofinputs, listoftargets)
        except:
            with console:
                print("Incorrect training list format.")
    
    TrainButton.on_click(TrainFunction)
    
    
    with out5:
        display(Trainboxestop, Trainboxesbottom)
EpochSaves = widgets.IntRangeSlider(
        min=1,
        max=1,
        value=[1,1],
        layout=widgets.Layout(width='300px')
)

#Creates saving widgets

def createSave(neuralnetwork):
    with out6:
        clear_output()
        printmd('### **Save**')
        
    SaveOutput1 = widgets.Textarea(
        value="",
        placeholder='',
        description='JSON:',
        disabled=False,
        layout = widgets.Layout(width='400px')
    )
    
    SaveOutput2 = widgets.Textarea(
        value="",
        placeholder='',
        description='Weights:',
        disabled=False,
        layout = widgets.Layout(width='400px')
    )
    
    EpochSavesWidge = widgets.HBox([
        widgets.Label(value="Epoch Range:", layout=widgets.Layout(width='200px')),
        EpochSaves    
    ], layout=widgets.Layout(width = '600px'))
    
    TrainButton = widgets.Button(description = "Train", button_style='info')
    
    SaveButton = widgets.Button(description = "Save", button_style='info')
    
    SaveBox = widgets.HBox([SaveOutput1, SaveOutput2])
    SaveBoxLower = widgets.HBox([EpochSavesWidge, SaveButton])
    
    with out6:
        display(SaveBox)
        display(SaveBoxLower)
        
    def SaveFunction(b):
        SaveOutput1.value = str(neuralnetwork.toJSON(EpochSaves.value[0], EpochSaves.value[1]))
        SaveOutput2.value = str((np.stack(neuralnetwork.printWeights(EpochSaves.value[0], EpochSaves.value[1]))).tolist())     
    
    SaveButton.on_click(SaveFunction)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10065687263212407, 'fraction_numerical': 0.012592330126195863, 'mean_word_length': 2.7758567459092314, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '175406', 'license': 'MIT', 'n_tokens_mistral': 10548, 'n_tokens_neox': 10129, 'n_words': 3053, 'path': 'EffortlessAI.ipynb', 'repo_name': 'dandip/EffortlessAI', 'size': '36698'}