<jupyter_start><jupyter_text>In order to successfully complete this assignment you need to participate both individually and in groups during class on **Wednesday February 26**.  In-Class Assignment: Artificial Neural NetworkFrom: [theconversation.com](http://theconversation.com/deep-learning-and-neural-networks-77259)In class today we summarize the steps involved in designing and training a feed-forward artificial neural network. We will use the python script file named [partSix.py](./partSix.py) file provided in the "Neural Networks Demystified" module which can be downloaded from github:    git clone https://github.com/stephencwelch/Neural-Networks-Demystified  Agenda for today's class (80 minutes) **Note:** This will likely take more than one day in class.  So we actually have 220 minutes. 1. [(10 minutes) Review pre-class assignment](Review_pre-class_assignment)1. [(20 minutes) Working with Scripts](Working_with_Scripts)1. [(20 minutes) Modify code to be more flexible](Modify_code_to_be_more_flexible)1. [(20 minutes) Use our ANN on the "Digits" dataset](ANN_and_Digits)1. [(30 minutes) Finding/Using Neural Networks Libraries](NN_Libraries) --- 1. Review pre-class assignment* [0225-BackProp-NN-pre-class-assignment](0225-BackProp-NN-pre-class-assignment.ipynb) --- 2. Working with ScriptsLets talk about working with scripts. The following example loads the partSix script into and imports everything. Although I hate this syntax, lets use it for now.<jupyter_code>from partSix import * #Reminder, your instructor hates this syntax
print("Input Data", X)
print("Output Data", y)
#Untrained Random Network
NN = Neural_Network()
y1 = NN.forward(X)
print("Untrained Output", y1)
#Training step
T = trainer(NN)
T.train(X,y)
#Trained Network
y2 = NN.forward(X)
print("Trained Output",y2)<jupyter_output>Trained Output [[0.74998719]
 [0.81997369]
 [0.92998639]]<jupyter_text>&9989; **DO THIS:** Calculate and compare the mean Euclidean error for untrained network (```y1```) and the trained network (```y2```).<jupyter_code>#Put your code here<jupyter_output><empty_output><jupyter_text>--- 3. Modify code to be more flexibleThe code for our Neural Network example above assumes an input layer size of 2, hidden layer size of 3 and an output layer size of 1.  We are going to now modify [partSix.py](partSix.py) to update the ```__init___``` function to take three inputs; input layers (```ILS```), output layers (```OLS```) and Hidden Layers (```HLS```). Step 1: code review&9989; **DO THIS:** As a class let's review the parts of the partSix.py file and see if we can understand what everything is doing. Step 2: rewrite/override the ```__init__``` function&9989; **DO THIS:** Lets try to improve the code in above so that the user can specify these as inputs when creating the Neural_Network object.  The new code should work such that the default values should stay the same. Rerun the above example to make sure it still works. The new syntax should be something like the following. The new defaults should make the following code work.<jupyter_code>NN = Neural_Network(ILS=64,OLS=1,HLS=3)<jupyter_output><empty_output><jupyter_text>**IMPORTANT SIDE NOTE:** it can be tricky modifying imported scripts and testing them inside of jupyter notebooks. The reason is that Jupyter will only import a package once.  Consider the following simple example:<jupyter_code>%%writefile testimport.py

# Creat a really simple script called testimport.py
print("importing testimport")<jupyter_output>Writing testimport.py<jupyter_text>The above cell creates a file called [testimport.py](testimport.py) in your current directory. The following code will import (aka run) the file.<jupyter_code>import testimport
import testimport<jupyter_output><empty_output><jupyter_text>Notice how only one of the import commands will execute the print statement. This is because jupyter thinks that testimport.py is already imported and assumes that it did not change so does not bother importing it a second time.  If you modify a file you need to go to the "Kernel" menu and select "Restart & Clear Output" which will reset the entire Kernel. Then run the notebook again from scratch.The following line is a bash command that should delete the ```testimport.py``` file so it will not clutter your directory. Note a quick way to run bash commands from inside jupyter notebooks (actually python in general) is to use the exclamation mark ```!```.<jupyter_code>!rm testimport.py<jupyter_output><empty_output><jupyter_text>--- 4. Use our ANN on the "Digits" dataset.Here is the code copied from the Machine Learning Module which downloads the "digits" dataset and should separates it into training and testing sets. However there may be an error on your machine:<jupyter_code>%matplotlib inline
import matplotlib.pylab as plt
import numpy as np
import sklearn
from sklearn.datasets import fetch_lfw_people, load_digits
from sklearn.cross_validation import train_test_split

sk_data = load_digits();

#Cool slider to browse all of the images.
from ipywidgets import interact
def browse_images(images, labels, categories):
    n = len(images)
    def view_image(i):
        plt.imshow(images[i], cmap=plt.cm.gray_r, interpolation='nearest')
        plt.title('%s' % categories[labels[i]])
        plt.axis('off')
        plt.show()
    interact(view_image, i=(0,n-1))
browse_images(sk_data.images, sk_data.target, sk_data.target_names)


feature_vectors = sk_data.data
class_labels = sk_data.target
categories = sk_data.target_names

N, h, w = sk_data.images.shape
train_vectors, test_vectors, train_labels, test_labels = train_test_split(feature_vectors, class_labels, test_size=0.25, random_state=1)<jupyter_output><empty_output><jupyter_text>&9989; **DO THIS:** Assuming you get an error, Google the error and fix the above code. &9989; **DO THIS:** Now, assuming we got the above functions to work properly, the following code reformats the digets data into vectors we can use in our new neural network.<jupyter_code>train2 = np.ndarray((len(train_labels),1))
for i in range(len(train_labels)):
    train2[i] = train_labels[i]/10
train_labels = train2

test2 = np.ndarray((len(test_labels),1))
for i in range(len(test_labels)):
    test2[i] = test_labels[i]/10
test_labels = test2

test_vectors = test_vectors/256
train_vectors = train_vectors/256
#Run the training. 
NN = Neural_Network(ILS=64,OLS=1,HLS=3)
T = trainer(NN)
T.train(train_vectors, train_labels)
pred_labels = NN.forward(train_vectors)

print("Training Data error", np.sum(np.sqrt((train_labels - pred_labels)*(train_labels-pred_labels)))/len(train_vectors)*10)

pred_labels = NN.forward(test_vectors)

print("Testing Data error", np.sum(np.sqrt((test_labels - pred_labels)*(test_labels-pred_labels)))/len(test_vectors)*10)

def plot_gallery(images, true_titles, pred_titles, h, w, n_row=5, n_col=5):
    """Helper function to plot a gallery of portraits"""
    plt.figure(figsize=(1.8 * n_col, 2.4 * n_row))
    plt.subplots_adjust(bottom=0, left=.01, right=.99, top=.90, hspace=.35)
    for i in range(n_row * n_col):
        plt.subplot(n_row, n_col, i + 1)
        image = test_vectors[i].reshape((8,8))
        plt.imshow(image, cmap=plt.cm.gray_r)
        plt.title('Pred='+str(pred_titles[i]*10), size=9)
        plt.xlabel('Actual='+str(true_titles[i]*10), size=9)
        plt.xticks(())
        plt.yticks(())

plot_gallery(test_vectors, test_labels, pred_labels, h,w)<jupyter_output><empty_output><jupyter_text>&9989; **DO THIS:** Modify the number of input, hidden and output layers to get the best fit of the data. &9989; **QUESTION:** What is your best result? Put your answer here --- 5. Finding/Using Neural Networks LibrariesIn this section we will repeat both examples from above (Grades and Digits) using a python neural network library.  &9989; **DO THIS:** - As a group, find examples of neural network packages in python.   List links to the python libraries you found. &9989; **DO THIS:** - Pick a package (or packages) you find interesting and get them working in this notebook.  I suggest that each group member try to pick a different package and spend about 10 minutes trying to install and get it working. After about 10 minutes compare notes and pick the one the group will think is the easiest.  &9989; **QUESTION:** What package did you pick.  Please include any installation code needed.<jupyter_code># Put your installation code here<jupyter_output><empty_output><jupyter_text>&9989; **DO THIS:** - Create an example to demonstrate that the Neural Network is working.  Preferably use an example that comes with the provided NN Package.<jupyter_code># Put your example code here<jupyter_output><empty_output><jupyter_text>&9989; **DO THIS:** - Reproduce the results from the "Grade" example above using ```X``` and ```y```:<jupyter_code># Put your Grade example code here<jupyter_output><empty_output><jupyter_text>&9989; **DO THIS:** - Reproduce the results from the "Digits" example above:<jupyter_code># Put your Digits example code here
print("Testing Data error", np.sum(np.sqrt((test_labels - pred_labels)*(test_labels-pred_labels)))/len(test_vectors))
def plot_gallery(images, true_titles, pred_titles, h, w, n_row=5, n_col=5):
    """Helper function to plot a gallery of portraits"""
    plt.figure(figsize=(1.8 * n_col, 2.4 * n_row))
    plt.subplots_adjust(bottom=0, left=.01, right=.99, top=.90, hspace=.35)
    for i in range(n_row * n_col):
        plt.subplot(n_row, n_col, i + 1)
        image = test_vectors[i].reshape((8,8))
        plt.imshow(image, cmap=plt.cm.gray_r)
        plt.title('Pred='+str(pred_titles[i]), size=9)
        plt.xlabel('Actual='+str(true_titles[i]), size=9)
        plt.xticks(())
        plt.yticks(())

plot_gallery(test_vectors, test_labels, pred_labels, h,w)

##ANSWER##<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10502236681577877, 'fraction_numerical': 0.02175681171207808, 'mean_word_length': 5.27359693877551, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '442474', 'license': 'MIT', 'n_tokens_mistral': 3212, 'n_tokens_neox': 2991, 'n_words': 1081, 'path': 'cmse802-s20/0226-BackProp-NN-in-class-assignment.ipynb', 'repo_name': 'Diane1306/cmse802_git_CompModelling', 'size': '9836'}