<jupyter_start><jupyter_text>Self-Driving Car Engineer Nanodegree Deep Learning Project: Build a Traffic Sign Recognition Classifier See 'P3 Writeup.md' for a writeup of this project and 'Traffic_Sign_Classifier.html' for a published document --- Step 0: Load The Data<jupyter_code>## Check that GPU environment is working
from tensorflow.python.client import device_lib
print(device_lib.list_local_devices())
# Load pickled data
import pickle
from scipy import stats

training_file = r'C:\Users\HalleyandWoody\Documents\Python\self-driving\P3-Traffic-Sign-Classifier-Project\traffic-signs-data\train.p'
validation_file= r'C:\Users\HalleyandWoody\Documents\Python\self-driving\P3-Traffic-Sign-Classifier-Project\traffic-signs-data\valid.p'
testing_file = r'C:\Users\HalleyandWoody\Documents\Python\self-driving\P3-Traffic-Sign-Classifier-Project\traffic-signs-data\test.p'

with open(training_file, mode='rb') as f:
    train = pickle.load(f)
with open(validation_file, mode='rb') as f:
    valid = pickle.load(f)
with open(testing_file, mode='rb') as f:
    test = pickle.load(f)
    
X_train, y_train = train['features'], train['labels']
X_valid, y_valid = valid['features'], valid['labels']
X_test, y_test = test['features'], test['labels']

### The following print statements are used to confirm that I understand
### The datasets I am dealing with. The next uncommented line uses 
### stats.describe from scipy in order to get more stats about our data.

# print(train['features'][0][0][0][0].shape)
# print(train['features'][34332].shape)
# print(train['features'][0].shape)
# print(train['features'].shape)
# print(valid['features'].shape)
# print(test['features'].shape)
# print(valid.keys())
# print(test.keys())
# print(train['labels'].shape)
# print(train['sizes'].shape)
# print(train['coords'].shape
# print(train['coords'].shape)
# stats.describe(X_train[0])
# print(X_train[0].shape)<jupyter_output><empty_output><jupyter_text>--- Step 1: Dataset Summary & ExplorationThe pickled data is a dictionary with 4 key/value pairs:- `'features'` is a 4D array containing raw pixel data of the traffic sign images, (num examples, width, height, channels).- `'labels'` is a 1D array containing the label/class id of the traffic sign. The file `signnames.csv` contains id -> name mappings for each id.- `'sizes'` is a list containing tuples, (width, height) representing the original width and height the image.- `'coords'` is a list containing tuples, (x1, y1, x2, y2) representing coordinates of a bounding box around the sign in the image. **THESE COORDINATES ASSUME THE ORIGINAL IMAGE. THE PICKLED DATA CONTAINS RESIZED VERSIONS (32 by 32) OF THESE IMAGES**Complete the basic data summary below. Use python, numpy and/or pandas methods to calculate the data summary rather than hard coding the results. For example, the [pandas shape method](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.shape.html) might be useful for calculating some of the summary results.<jupyter_code>#Get stats about training image pixels
stats.describe(X_train)
#Get stats about training images' labels
stats.describe(y_train)
#Get stats about training images' original sizes
stats.describe(train['sizes'])
#Get stats about training images' sign bounding boxes
stats.describe(train['coords'])
#Get stats about validation image pixels
stats.describe(X_valid)
#Get stats about validation image labels
stats.describe(y_valid)
#Get stats about validation set images' original sizes
stats.describe(valid['sizes'])
#Get stats about validation set images' sign bounding boxes
stats.describe(valid['coords'])
#Get stats about testing set image pixels
stats.describe(X_test)
#Get stats about testing set image labels
stats.describe(y_test)
#Get stats about testing set images' original sizes
stats.describe(test['sizes'])
#Get stats about testing set images' sign bounding box
stats.describe(test['coords'])<jupyter_output><empty_output><jupyter_text>Provided below is a Basic Summary of the Data Set Using Python, Numpy and/or Pandas<jupyter_code># Number of training examples
n_train = train['labels'].shape[0]

# Number of validation examples
n_validation = valid['labels'].shape[0]

# Number of testing examples.
n_test = test['labels'].shape[0]

# What's the shape of an traffic sign image?
image_shape = train['features'][0].shape

# Finding class count
L = []

for e in train['labels']:
    if e not in L:
        L.append(e)

n_classes = len(L)

print("Number of training examples =", n_train)
print("Number of validation examples =", n_validation)
print("Number of testing examples =", n_test)
print("Image data shape =", image_shape)
print("Number of classes =", n_classes)<jupyter_output>Number of training examples = 34799
Number of validation examples = 4410
Number of testing examples = 12630
Image data shape = (32, 32, 3)
Number of classes = 43<jupyter_text>Include an exploratory visualization of the dataset Visualize the German Traffic Signs Dataset using the pickled file(s). This is open ended, suggestions include: plotting traffic sign images, plotting the count of each sign, etc. The [Matplotlib](http://matplotlib.org/) [examples](http://matplotlib.org/examples/index.html) and [gallery](http://matplotlib.org/gallery.html) pages are a great resource for doing visualizations in Python.**NOTE:** It's recommended you start with something simple first. If you wish to do more, come back to it after you've completed the rest of the sections. It can be interesting to look at the distribution of classes in the training, validation and test set. Is the distribution the same? Are there more examples of some classes than others?<jupyter_code>import matplotlib.pyplot as plt; plt.rcdefaults()
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg

import os,sys


    
# To explore the data, I decided to show histograms for each set (training, test and validation) that represent the frequency of each label (0 to 42)
# This can be beneficial later on in order to see whether prediction accuracy is label dependent
# in case we decide to look more closely into our model prediction accuracy 

# Number of bins is equal to number of labels
n_bins = 43

# Create lists with frequency of each label for all three sets (training, test, validation)
test_qty = []
for e in test['labels']:
    test_qty.append(e)

train_qty = []
for e in train['labels']:
    train_qty.append(e)
    
valid_qty = []
for e in valid['labels']:
    valid_qty.append(e)

# Visualizing test set
fig, axs = plt.subplots(1, figsize=(20,5))
# We can set the number of bins with the `bins` kwarg
axs.hist(test_qty, bins=n_bins)

#showing all ticks to make it easier to read
plt.xticks(np.arange(0, 43, 1.0))
plt.show()
# # Visualizations will be shown in the notebook.
%matplotlib inline
fig.savefig('test_labels.png')

# Following 7 lines of code modify the extension to .jpg since I was 
# having trouble viewing .png images in my .md write up file
folder = r'C:\Users\HalleyandWoody\Documents\Python\self-driving\P3-Traffic-Sign-Classifier-Project'
infilename = os.path.join(folder,'test_labels.png')
oldbase = os.path.splitext('test_labels.png')
newname = infilename.replace('.png', '.jpg')
try:
    output = os.rename(infilename, newname)
except:
    pass

# Showing training set data

fig, axs = plt.subplots(1, figsize=(20,5))
axs.hist(train_qty, bins=n_bins)

#showing all ticks to make it easier to read
plt.xticks(np.arange(0, 43, 1.0))

plt.show()
# # Visualizations will be shown in the notebook.
%matplotlib inline

fig.savefig('training_labels.png')

# Following 7 lines of code modify the extension to .jpg since I was 
# having trouble viewing .png images in my .md write up file
infilename = os.path.join(folder,'training_labels.png')
oldbase = os.path.splitext('training_labels.png')
newname = infilename.replace('.png', '.jpg')
try:
    output = os.rename(infilename, newname)
except:
    pass
# Showing validation set data

fig, axs = plt.subplots(1, figsize=(20,5))
axs.hist(valid_qty, bins=n_bins)

#showing all ticks to make it easier to read
plt.xticks(np.arange(0, 43, 1.0))

plt.show()
# # Visualizations will be shown in the notebook.
%matplotlib inline

fig.savefig('validation_labels.png')

# Following 7 lines of code modify the extension to .jpg since I was 
# having trouble viewing .png images in my .md write up file
infilename = os.path.join(folder,'validation_labels.png')
oldbase = os.path.splitext('validation_labels.png')
newname = infilename.replace('.png', '.jpg')
try:
    output = os.rename(infilename, newname)
except:
    pass<jupyter_output><empty_output><jupyter_text>---- Step 2: Design and Test a Model ArchitectureDesign and implement a deep learning model that learns to recognize traffic signs. Train and test your model on the [German Traffic Sign Dataset](http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset).The LeNet-5 implementation shown in the [classroom](https://classroom.udacity.com/nanodegrees/nd013/parts/fbf77062-5703-404e-b60c-95b78b2f3f9e/modules/6df7ae49-c61c-4bb2-a23e-6527e69209ec/lessons/601ae704-1035-4287-8b11-e2c2716217ad/concepts/d4aca031-508f-4e0b-b493-e7b706120f81) at the end of the CNN lesson is a solid starting point. You'll have to change the number of classes and possibly the preprocessing, but aside from that it's plug and play! With the LeNet-5 solution from the lecture, you should expect a validation set accuracy of about 0.89. To meet specifications, the validation set accuracy will need to be at least 0.93. It is possible to get an even higher accuracy, but 0.93 is the minimum for a successful project submission. There are various aspects to consider when thinking about this problem:- Neural network architecture (is the network over or underfitting?)- Play around preprocessing techniques (normalization, rgb to grayscale, etc)- Number of examples per label (some have more than others).- Generate fake data.Here is an example of a [published baseline model on this problem](http://yann.lecun.com/exdb/publis/pdf/sermanet-ijcnn-11.pdf).   Pre-process the Data Set (normalization, grayscale, etc.) Minimally, the image data should be normalized so that the data has mean zero and equal variance. For image data, `(pixel - 128)/ 128` is a quick way to approximately normalize the data and can be used in this project. Other pre-processing steps are optional. You can try different techniques to see if it improves performance. Use the code cell (or multiple code cells, if necessary) to implement the first step of your project.<jupyter_code>### Preprocessing the data  
import cv2

# Next lines of code until line 11 visualize a RGB image (OR BGR, as read by OpenCV), its conversion to grayscale 
# and some more pre-processing: I decided to resize the image back to its original size (not 32x32) in order to 
# properly trim it using the values stored in the 'coords' key. After that, I resized the image back to 32x32
# so that it works with the LeNet5 architecture

gray_img = cv2.cvtColor(train['features'][1000], cv2.COLOR_BGR2GRAY)


# gray_img1 = cv2.resize(gray_img,(train['sizes'][1000][0],train['sizes'][1000][1]))
# gray_img1 = gray_img1[train['coords'][1000][0]:train['coords'][1000][2],train['coords'][1000][1]:train['coords'][1000][3]]
# gray_img1 = cv2.resize(gray_img1,(32,32))



f = plt.figure()
f.add_subplot(1,2, 1)
plt.imshow(train['features'][1000])
f.add_subplot(1,2, 2)
plt.imshow(gray_img, cmap = plt.get_cmap('gray'))
# f.add_subplot(1,3, 3)
# plt.imshow(gray_img1, cmap = plt.get_cmap('gray'))

f.savefig('grayscale_example.png')

# Creates a list to append normalized image data to. I realized that I cannot alter the images as is
# Since it wouldn't make sense for an image to store negative values (so it doesn't, it returns either 0 or 255 instead
# which prevents normalization)
L_train = []
L_train_new_data = []
L_new_labels = []

for i in range(0, n_train):
#     print(L[0])
    D3 = cv2.cvtColor(train['features'][i], cv2.COLOR_BGR2GRAY) #CONVERT TO GRAYSCALE
#     D4 = cv2.resize(D3,(train['sizes'][i][0],train['sizes'][i][1])) #resize to unprocessed size
#     D4 = D4[train['coords'][i][0]:train['coords'][i][2],train['coords'][i][1]:train['coords'][i][3]] #trim outside bounding box
#     D4 = cv2.resize(D4,(32,32))
    
    D3 = D3[:,:,None] # Slicing to maintain the three dimensional image shape
#     D4 = D4[:,:,None] # Slicing to maintain the three dimensional image shape

#     D3 = train['features'][i]
    L_train.append(D3)
    L_train[-1] = (L_train[-1]-128.0)/128.0 #normalize
    

print(n_train == len(L_train)) # validate that length of list equals n_validation


L_validation = []
for i in range(0, n_validation):
#     print(L[0])
    D3 = cv2.cvtColor(valid['features'][i], cv2.COLOR_BGR2GRAY) #CONVERT TO GRAYSCALE
#     D4 = cv2.resize(D3,(valid['sizes'][i][0],valid['sizes'][i][1])) #resize to unprocessed size
#     D4 = D3[valid['coords'][i][0]:valid['coords'][i][2],valid['coords'][i][1]:valid['coords'][i][3]] #trim outside bounding box
#     D4 = cv2.resize(D3,(32,32))
    D3 = D3[:,:,None] # Slicing to maintain the three dimensional image shape
    
    L_validation.append(D3)    
    L_validation[-1] = (L_validation[-1]-128.0)/128.0
#     print(L[0])
print(n_validation == len(L_validation)) # validate that length of list equals n_validation

L_test = []
for i in range(0, n_test):
#     print(L[0])
    D3 = cv2.cvtColor(test['features'][i], cv2.COLOR_BGR2GRAY) #CONVERT TO GRAYSCALE
    D3 = D3[:,:,None] # Slicing to maintain the three dimensional image shape
    
#     D3 = test['features'][i]
    L_test.append(D3)
    L_test[-1] = (L_test[-1]-128.0)/128.0
#     print(L[0])
print(n_test == len(L_test)) # validate that length of list equals n_test<jupyter_output>True
True
True<jupyter_text>Model Architecture<jupyter_code>from sklearn.utils import shuffle
import tensorflow as tf
from tensorflow.contrib.layers import flatten

# Shuffle the training data (normalized features (image data) and their corresponding labels) in order to 
# keep training general and prevent overfitting.

L_train, train['labels'] = shuffle(L_train, train['labels'])
EPOCHS = 50
BATCH_SIZE = 128<jupyter_output><empty_output><jupyter_text>Implementing LeNet-5 architecture Important formulas:W = width, F = convolution filter size, P = padding (0 for 'valid' and 1 for 'same'), S = stridewidth of the next layer: W_out =[ (W−F+2P)/S] + 1.The output height: H_out = [(H-F+2P)/S] + 1.output depth is equal to the number of filters: D_out = K. InputThe LeNet architecture accepts a 32x32xC image as input, where C is the number of color channels.  Architecture**Layer 1: Convolutional.** The output shape should be 28x28x6.**Activation.** RELU activation function**Pooling.** The output shape should be 14x14x6.**Layer 2: Convolutional.** The output shape should be 10x10x16.**Activation.** RELU activation function**Pooling.** The output shape should be 5x5x16.**Flatten.** Flatten the output shape of the final pooling layer such that it's 1D instead of 3D. The easiest way to do is by using `tf.contrib.layers.flatten`.**Layer 3: Fully Connected.** This should have 120 outputs.**Activation.** RELU activation function**Layer 4: Fully Connected.** This should have 84 outputs.**Activation.** RELU activation function**Drop out** Drop out some of the data depending on the value of 'prob' below **Layer 5: Fully Connected (Logits).** This should have 43 outputs.<jupyter_code>def LeNet(x):    
    # Arguments used for tf.truncated_normal, randomly defines variables for the weights and biases for each layer
    mu = 0
    sigma = 0.1
        
    # Layer 1: Convolutional. Input = 32x32x1. Output = 28x28x6.
    conv1_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 1, 6), mean = mu, stddev = sigma))
    conv1_b = tf.Variable(tf.zeros(6))
    conv1   = tf.nn.conv2d(x, conv1_W, strides=[1, 1, 1, 1], padding='VALID') + conv1_b
    
#     print('conv1 = ' + str(conv1))
#     print('x = ' + str(x))

    # Activation.
    conv1 = tf.nn.relu(conv1)
#     print('conv1 act = ' + str(conv1))

    # Pooling. Input = 28x28x6. Output = 14x14x6.
    conv1 = tf.nn.max_pool(conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')
    LeNet.conv1 = conv1 #make conv1 accessible from another function
#     print('pool1 = ' + str(conv1))

    
    
    # Layer 2: Convolutional. Output = 10x10x16.
    conv2_W = tf.Variable(tf.truncated_normal(shape=(5, 5, 6, 16), mean = mu, stddev = sigma))
    conv2_b = tf.Variable(tf.zeros(16))
    conv2   = tf.nn.conv2d(conv1, conv2_W, strides=[1, 1, 1, 1], padding='VALID') + conv2_b
    
    # Activation.
    conv2 = tf.nn.relu(conv2)

    # Pooling. Input = 10x10x16. Output = 5x5x16.
    conv2 = tf.nn.max_pool(conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')

    # Flatten. Input = 5x5x16. Output = 400.
    fc0   = flatten(conv2)
    
    # Layer 3: Fully Connected. Input = 400. Output = 120.
    fc1_W = tf.Variable(tf.truncated_normal(shape=(400, 120), mean = mu, stddev = sigma))
    fc1_b = tf.Variable(tf.zeros(120))
    fc1   = tf.matmul(fc0, fc1_W) + fc1_b
    
    # Activation.
    fc1    = tf.nn.relu(fc1)

    # Layer 4: Fully Connected. Input = 120. Output = 84.
    fc2_W  = tf.Variable(tf.truncated_normal(shape=(120, 84), mean = mu, stddev = sigma))
    fc2_b  = tf.Variable(tf.zeros(84))
    fc2    = tf.matmul(fc1, fc2_W) + fc2_b
    
    # Activation.
    fc2    = tf.nn.relu(fc2)
    
    #Drop out
#     prob = tf.placeholder_with_default(1.0, shape=())
    fc2  = tf.nn.dropout(fc2, prob)

    # Layer 5: Fully Connected. Input = 84. Output = 43.
    fc3_W  = tf.Variable(tf.truncated_normal(shape=(84, 43), mean = mu, stddev = sigma))
    fc3_b  = tf.Variable(tf.zeros(43))
    logits = tf.matmul(fc2, fc3_W) + fc3_b
    
    
    
    return logits<jupyter_output><empty_output><jupyter_text>Train, Validate and Test the Model A validation set was used to assess how well the model is performing. A low accuracy on the training and validationsets imply underfitting. A high accuracy on the training set but low accuracy on the validation set implies overfitting.<jupyter_code>### Training the model here.
### Calculate and report the accuracy on the training and validation set.

prob = tf.placeholder_with_default(1.0, shape=())
x = tf.placeholder(tf.float32, (None, 32, 32, 1)) # placeholder for images
y = tf.placeholder(tf.int32, (None)) # placeholder for a batch of labels
one_hot_y = tf.one_hot(y, 43)

# Training setup. Setting up the learning rate, defining error function or cross entropy and minimizing (optimizing) 
#it using tensorflow's built in Adam optimizer

rate = 0.003

logits = LeNet(x)
print('logits' + str(logits.shape))
print(logits[0])
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=one_hot_y, logits=logits)
loss_operation = tf.reduce_mean(cross_entropy)
optimizer = tf.train.AdamOptimizer(learning_rate = rate)
training_operation = optimizer.minimize(loss_operation)


# evaluate() finds the accuracy of the model for an input dataset

correct_prediction = tf.equal(tf.argmax(logits, 1), tf.argmax(one_hot_y, 1))
accuracy_operation = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
saver = tf.train.Saver()

def evaluate(X_data, y_data):
    num_examples = len(X_data)
    total_accuracy = 0
    sess = tf.get_default_session()
    for offset in range(0, num_examples, BATCH_SIZE):
        batch_x, batch_y = X_data[offset:offset+BATCH_SIZE], y_data[offset:offset+BATCH_SIZE]
        accuracy = sess.run(accuracy_operation, feed_dict={x: batch_x, y: batch_y})
        total_accuracy += (accuracy * len(batch_x))
    return total_accuracy / num_examples
    
# Running the training data through the training pipeline to train the model.
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    num_examples = len(X_train)
    
    print("Training...")
#     print()
    for i in range(EPOCHS):
        #before each EPOCH, shuffle the training set
        L_train, train['labels'] = shuffle(L_train, train['labels']) 
        for offset in range(0, num_examples, BATCH_SIZE):
            end = offset + BATCH_SIZE
            batch_x, batch_y = L_train[offset:end], train['labels'][offset:end]
            sess.run(training_operation, feed_dict={x: batch_x, y: batch_y, prob: 0.5})

#         After each epoch, measure the loss and accuracy of the validation set.
#         training_accuracy = evaluate(L_train, train['labels'])
        validation_accuracy = evaluate(L_validation, valid['labels'])
        train_accuracy = evaluate(L_train, train['labels'])

        print("EPOCH {} ...".format(i+1))
        print("Validation Accuracy = {:.3f}".format(validation_accuracy))
        print("Training Accuracy = {:.3f}".format(train_accuracy))

            

    
    # saving the model 

    saver.save(sess, './lenet')
    print("Model saved")
    


with tf.Session() as sess:
    saver.restore(sess, tf.train.latest_checkpoint('.'))

    test_accuracy = evaluate(L_test, test['labels'])
    print("Test Accuracy = {:.3f}".format(test_accuracy))<jupyter_output>INFO:tensorflow:Restoring parameters from .\lenet
Test Accuracy = 0.924<jupyter_text>--- Step 3: Test a Model on New ImagesTo give more insight into how the model is working, I downloaded five pictures of German traffic signs from the web and used the model to predict the traffic sign type.The file `signnames.csv` is useful as it contains mappings from the class id (integer) to the actual sign name.  Load and Output the Images<jupyter_code>### Load the images and plot them.

img1 = cv2.imread('German-traffic-signs\id4.png')
img2 = cv2.imread('German-traffic-signs\id14.png')
img3 = cv2.imread('German-traffic-signs\id25.png')
img4 = cv2.imread('German-traffic-signs\id27.png')
img5 = cv2.imread('German-traffic-signs\id36.png')

image_list = []

#resize images to 32x32
for e in [img1,img2,img3,img4,img5]:
    e = cv2.cvtColor(e, cv2.COLOR_BGR2RGB)
    e = cv2.resize(e,(32,32))
    image_list.append(e)
    
labels1 = [4,14,25,27,36]



f = plt.figure()

# Show processed 32x32 images in RGB mode
for i in range(len(image_list)):
    f.add_subplot(1,5, i+1)
    plt.imshow(image_list[i], cmap = plt.get_cmap('gray'))

## the 4th image from the left seems particularly hard to classify since the sign is not parallel to the camera lens
## (Transposing the image could be useful)<jupyter_output><empty_output><jupyter_text>Predict the Sign Type for Each Image<jupyter_code>### Run the predictions here and use the model to output the prediction for each image.
### Make sure to pre-process the images with the same pre-processing pipeline used earlier.
### Feel free to use as many code cells as needed.

processed_images = []
for i in range(0, len(image_list)):
    D3 = cv2.cvtColor(image_list[i], cv2.COLOR_BGR2GRAY) #CONVERT TO GRAYSCALE
#     D4 = cv2.resize(D3,(valid['sizes'][i][0],valid['sizes'][i][1])) #resize to unprocessed size
#     D4 = D3[valid['coords'][i][0]:valid['coords'][i][2],valid['coords'][i][1]:valid['coords'][i][3]] #trim outside bounding box
#     D4 = cv2.resize(D3,(32,32))
    D3 = D3[:,:,None] # Slicing to maintain the three dimensional image shape
    
    processed_images.append(D3)    
    processed_images[-1] = (processed_images[-1]-128.0)/128.0
#     print(L[0])
print(n_validation == len(L_validation)) # validate that length of list equals n_validation

with tf.Session() as sess:
    saver.restore(sess, tf.train.latest_checkpoint('.'))

    test_accuracy1 = evaluate(processed_images[0][None,:,:], ([labels1[0]]))
    test_accuracy2 = evaluate(processed_images[1][None,:,:], ([labels1[1]]))
    test_accuracy3 = evaluate(processed_images[2][None,:,:], ([labels1[2]]))
    test_accuracy4 = evaluate(processed_images[3][None,:,:], ([labels1[3]]))
    test_accuracy5 = evaluate(processed_images[4][None,:,:], ([labels1[4]]))


    print("Test Accuracy 1 = {:.3f}".format(test_accuracy1))
    print("Test Accuracy 2 = {:.3f}".format(test_accuracy2))
    print("Test Accuracy 3 = {:.3f}".format(test_accuracy3))
    print("Test Accuracy 4 = {:.3f}".format(test_accuracy4))
    print("Test Accuracy 5 = {:.3f}".format(test_accuracy5))<jupyter_output>True
INFO:tensorflow:Restoring parameters from .\lenet
Test Accuracy 1 = 1.000
Test Accuracy 2 = 0.000
Test Accuracy 3 = 1.000
Test Accuracy 4 = 0.000
Test Accuracy 5 = 1.000<jupyter_text>Analyze Performance<jupyter_code>### Calculate the accuracy for these 5 new images. 

accuracy = (test_accuracy1 + test_accuracy2 +test_accuracy3 + test_accuracy4 + test_accuracy5)*100/5
print(str(accuracy) + '%')<jupyter_output>60.0%<jupyter_text>Output Top 5 Softmax Probabilities For Each Image Found on the Web<jupyter_code>### Print out the top five softmax probabilities for the predictions on the German traffic sign images found on the web. 
### Feel free to use as many code cells as needed.
 
print(labels1) 
with tf.Session() as sess: 
    saver.restore(sess, tf.train.latest_checkpoint('.'))
    probs = sess.run(tf.nn.softmax(logits), feed_dict={x: processed_images})
    top5 = sess.run(tf.nn.top_k(tf.constant(probs), k = 5))
    print("Softmax Probabilities: ")
    print(top5)

    
# The model is very certain when it guessed right (almost 100%). It is also very certain when
# it guessed wrong (top softmax probability is at least ten orders of magnitude higher than
# the second probability)<jupyter_output>[4, 14, 25, 27, 36]
INFO:tensorflow:Restoring parameters from .\lenet
Softmax Probabilities: 
TopKV2(values=array([[1.0000000e+00, 1.7068588e-26, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00],
       [1.0000000e+00, 1.2158862e-14, 5.9377502e-22, 3.1331551e-22,
        1.8155302e-24],
       [1.0000000e+00, 1.5043357e-31, 1.1699521e-31, 3.3121129e-37,
        3.3995271e-38],
       [9.9999988e-01, 1.5213770e-07, 4.2433603e-09, 1.7506623e-14,
        4.5339958e-16],
       [1.0000000e+00, 1.8505612e-17, 1.3485954e-17, 8.3097870e-18,
        2.2366852e-18]], dtype=float32), indices=array([[ 4,  5,  0,  1,  2],
       [12,  1, 11,  8, 40],
       [25, 29, 31, 22,  5],
       [38,  1, 11,  3, 18],
       [36,  3, 25, 13,  6]]))
{'fraction_non_alphanumeric': 0.1228787647148754, 'fraction_numerical': 0.042615807980431124, 'mean_word_length': 4.479581151832461, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 44, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 7, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '634850', 'license': 'MIT', 'n_tokens_mistral': 9440, 'n_tokens_neox': 8664, 'n_words': 2661, 'path': 'Traffic_Sign_Classifier.ipynb', 'repo_name': 'woodyhaddad/SDCND-1-P3-Traffic-Sign-Classifier', 'size': '26164'}