<jupyter_start><jupyter_text>Assignment 5: Traffic Sign Classifier---Using computer vision, deep learning, and convolutional neural networks, create a pipeline that classifies traffic signs.  Train and validate a model so it can classify traffic sign images using the [German Traffic Sign Dataset](http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset). After the model is trained, test it on images of traffic signs that you find on the web.The goals of this project are the following:- Load the data set- Explore, summarize and visualize the data set- Design, train and test a model architecture- Use the model to make predictions on new images- Use data augmentation and transfer learning- Analyze the softmax probabilities of the new images- Summarize the results of each step with a written reportThe writeup should be included in additional cells within the notebook. Some sample cells are included so that you have a guide of what is expected of the writeup. Additionally, add a README file with an introduction to what you developed, as well as the steps needed to run it.**Notes:** - Unless you have set up a GPU on a desktop with all the needed packages, train the model using a GPU enabled google colab environment.- Once you have completed all of the code implementations, you need to finalize your work by exporting the iPython Notebook as an HTML document. Before exporting the notebook to html, all of the code cells need to have been run.**Rubric:**1. Uploaded the notbook with all cells executed and the writeup cells, and the HTML output of the code.2. Downloaded and loaded the dataset. There was a description of the data, and a visualization of one image of each category.3. Preprocessed the data, and provided a description of the preprocessing steps taken. Use data augmentation to improve the available data.4. Defined a model architecture using Convlolutional layers. Justified the model architecture based on other models, such as VGG or GoogleNet.5. Described how the model was trained, mentioning how the hyperparameters (optimizer, batch size, loss function, etc) were selected, and what tests / modifications to the model you performed.6. Tested the model on a subset of the data set and measure performance. 7. Included images of traffic signs found from the web in a different folder, and tested the model on said images. Those images were visualized, and a discussion as to why those were chosen was included.8. Measured the performance on the new images and compared it to the test dataset.9. For the new images, output of the top 5 softmax predictions was included, and mentioned why you think those predictions were made.10. Achieved at least 80% accuracy on the test images.11. Create an additional model using transfer learning to try to improve performance.12. Saved the model and weights to a file for later use.  Basic includes<jupyter_code>import numpy as np

import matplotlib.pyplot as plt
import cv2

from keras import models, layers, optimizers
from keras.preprocessing.image import ImageDataGenerator

import os<jupyter_output><empty_output><jupyter_text>Load and visualize the dataProvide a basic summary of the German traffic sign data set. In the code, the analysis should be done using python, numpy, and matplotlib rather than hardcoding results manually. Include a graph of the distribution of the image classes.- The size of training set is ?- The size of the validation set is ?- The size of test set is ?- The shape of a traffic sign image is ?- The number of unique classes/labels in the data set is ?  Load the dataset<jupyter_code># TODO: Add the needed code to load data set
train_dir = pass
test_dir = pass

train_data, train_labels = pass
test_data, test_labels = pass<jupyter_output><empty_output><jupyter_text>Visualize the data setUsing matplotlib, include a figure with one image from each class. Propose a class name for each.<jupyter_code># TODO: Add the needed code to visualize<jupyter_output><empty_output><jupyter_text>Preprocess the data setPropose a preprocessing pipeline for the images. At least, the data should be normalized and augmented, but other techniques have to be considered: grayscale, blurring, using mean and standard deviation, etc. Describe what techniques you used and why. Consider using both OpenCV and [Keras' Imagedatagenerators](https://keras.io/preprocessing/image/) for this step.Additionally, include a plot of the images before and after the preprocessing step.<jupyter_code># TODO: Preprocess the data<jupyter_output><empty_output><jupyter_text>Design and test a modelDesign and implement a deep learning model, tran and test it with the german traffic sign data set. It is recommended to start from already stablished models. For reference, look at: [Common architectures in convolutional neural networks.](https://www.jeremyjordan.me/convnet-architectures/), [Convolutional networks case studies](http://cs231n.github.io/convolutional-networks/case), [VGG Net, 1st runner of of ilsvlc 2014](https://medium.com/coinmonks/paper-review-of-vggnet-1st-runner-up-of-ilsvlc-2014-image-classification-d02355543a11), [GoogLeNet, winner of ilsvrc 2014](https://medium.com/coinmonks/paper-review-of-googlenet-inception-v1-winner-of-ilsvlc-2014-image-classification-c2b3565a64e7), [Resnet, winner of ilsvrc 2015](https://towardsdatascience.com/review-resnet-winner-of-ilsvrc-2015-image-classification-localization-detection-e39402bfa5d8).Describe the model you used and discuss why you selected it.<jupyter_code># TODO: Define a model based on a particular architecture. Include the model summary.<jupyter_output><empty_output><jupyter_text>Train and test the modelInclude a discussion on the tests / different configurations used for the optimizer, batch size, epochs, and other hyperparameters for the training. Use early stopping and save the best weights as the training develops. Include plots for the training and validation accuracy of the model. Additionally, evaluate the model with the test data. If you used a stablished architecture, mention why you think it was suitable for this problem.- training set accuracy of ?- validation set accuracy of ?- test set accuracy of ?- What were some problems with the initial architecture?- How was the architecture adjusted and why was it adjusted? - Which parameters were tuned? How were they adjusted and why?- Did the model over or underfit? When and why do you think it happened?After the training step, save both the model and the weights to a file.<jupyter_code># TODO: Define the training, and discuss the process<jupyter_output><empty_output><jupyter_text>Test the model on new imagesFind at least 10 new images from different categories and use your model to predict the traffic sign type. Plot each of the images before the prediction, after the preprocessing step, and after the prediction with its predicted label.  Load the images<jupyter_code># TODO Load the new images<jupyter_output><empty_output><jupyter_text>Predict the type for each image<jupyter_code># TODO use the model to make prediction on the images. Remember to preprocess the new images with 
# the same process used for the training images<jupyter_output><empty_output><jupyter_text>Analyze the performanceCalculate the accuracy, and plot the images that were incorrectly classified. Discuss why those images were incorrectly classified.<jupyter_code># TODO Calculate the accuracy for the new images<jupyter_output><empty_output><jupyter_text>Output the top 5 softmax probabilites for each imageFor each of the new images, print the top 5 softmax categories it predicted. Check [numpy argsort](https://docs.scipy.org/doc/numpy/reference/generated/numpy.argsort.html) to get sorted indices of a prediction.<jupyter_code># TODO: for each image, plot the image and print the top 5 softmax categories predicted<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05326722603610475, 'fraction_numerical': 0.010170353419781338, 'mean_word_length': 5.243650793650794, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 6, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '467278', 'license': 'MIT', 'n_tokens_mistral': 2047, 'n_tokens_neox': 1952, 'n_words': 1052, 'path': 'Assignment_Traffic_sign_classifier.ipynb', 'repo_name': 'DeepLearningVision-2019/a5_traffic_sign_classifier', 'size': '7866'}