<jupyter_start><jupyter_text>Keras Intro: Convolutional ModelsKeras Documentation: https://keras.ioIn this notebook we explore how to use Keras to implement Convolutional models  Machine learning on images<jupyter_code>import pandas as pd
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>MNIST<jupyter_code>from keras.datasets import mnist
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train.shape
X_test.shape
plt.imshow(X_train[0], cmap='gray')
X_train_flat = X_train.reshape(-1, 28*28)
X_test_flat = X_test.reshape(-1, 28*28)
X_train_flat.shape
X_train_sc = X_train_flat.astype('float32') / 255.0
X_test_sc = X_test_flat.astype('float32') / 255.0
from keras.utils.np_utils import to_categorical
y_train_cat = to_categorical(y_train)
y_test_cat = to_categorical(y_test)
y_train[0]
y_train_cat[0]
y_train_cat.shape
y_test_cat.shape<jupyter_output><empty_output><jupyter_text>Fully connected on images<jupyter_code>from keras.models import Sequential
from keras.layers import Dense
import keras.backend as K

K.clear_session()

model = Sequential()
model.add(Dense(512, input_dim=28*28, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

h = model.fit(X_train_sc, y_train_cat, batch_size=128, epochs=10, verbose=1, validation_split=0.3)
plt.plot(h.history['acc'])
plt.plot(h.history['val_acc'])
plt.legend(['Training', 'Validation'])
plt.title('Accuracy')
plt.xlabel('Epochs')
test_accuracy = model.evaluate(X_test_sc, y_test_cat)[1]
test_accuracy<jupyter_output><empty_output><jupyter_text>Convolutional layers<jupyter_code>from keras.layers import Conv2D
from keras.layers import MaxPool2D
from keras.layers import Flatten, Activation
X_train_t = X_train_sc.reshape(-1, 28, 28, 1)
X_test_t = X_test_sc.reshape(-1, 28, 28, 1)
X_train_t.shape
K.clear_session()

model = Sequential()

model.add(Conv2D(32, (3, 3), input_shape=(28, 28, 1)))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Activation('relu'))

model.add(Flatten())

model.add(Dense(128, activation='relu'))

model.add(Dense(10, activation='softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])
model.summary()
model.fit(X_train_t, y_train_cat, batch_size=128,
          epochs=2, verbose=1, validation_split=0.3)
model.evaluate(X_test_t, y_test_cat)<jupyter_output><empty_output><jupyter_text>Exercise 1You've been hired by a shipping company to overhaul the way they route mail, parcels and packages. They want to build an image recognition system  capable of recognizing the digits in the zipcode on a package, so that it can be automatically routed to the correct location.You are tasked to build the digit recognition system. Luckily, you can rely on the MNIST dataset for the intial training of your model!Build a deep convolutional neural network with at least two convolutional and two pooling layers before the fully connected layer.- Start from the network we have just built- Insert a `Conv2D` layer after the first `MaxPool2D`, give it 64 filters.- Insert a `MaxPool2D` after that one- Insert an `Activation` layer- retrain the model- does performance improve?- how many parameters does this new model have? More or less than the previous model? Why?- how long did this second model take to train? Longer or shorter than the previous model? Why?- did it perform better or worse than the previous model?<jupyter_code>K.clear_session()

model = Sequential()

model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(MaxPool2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))

model.add(Flatten())

model.add(Dense(128, activation='relu'))

model.add(Dense(10, activation='softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])
model.summary()
model.fit(X_train_t, y_train_cat, batch_size=128,
          epochs=2, verbose=1, validation_split=0.3)
model.evaluate(X_test_t, y_test_cat)<jupyter_output><empty_output><jupyter_text>Exercise 2Pleased with your performance with the digits recognition task, your boss decides to challenge you with a harder task. Their online branch allows people to upload images to a website that generates and prints a postcard that is shipped to destination. Your boss would like to know what images people are loading on the site in order to provide targeted advertising on the same page, so he asks you to build an image recognition system capable of recognizing a few objects. Luckily for you, there's a dataset ready made with a collection of labeled images. This is the [Cifar 10 Dataset](http://www.cs.toronto.edu/~kriz/cifar.html), a very famous dataset that contains images for 10 different categories:- airplane 										- automobile 										- bird 										- cat 										- deer 										- dog 										- frog 										- horse 										- ship 										- truckIn this exercise we will reach the limit of what you can achieve on your laptop and get ready for the next session on cloud GPUs.Here's what you have to do:- load the cifar10 dataset using `keras.datasets.cifar10.load_data()`- display a few images, see how hard/easy it is for you to recognize an object with such low resolution- check the shape of X_train, does it need reshape?- check the scale of X_train, does it need rescaling?- check the shape of y_train, does it need reshape?- build a model with the following architecture, and choose the parameters and activation functions for each of the layers:    - conv2d    - conv2d    - maxpool    - conv2d    - conv2d    - maxpool    - flatten    - dense    - output- compile the model and check the number of parameters- attempt to train the model with the optimizer of your choice. How fast does training proceed?- If training is too slow (as expected) stop the execution and move to the next session!<jupyter_code>from keras.datasets import cifar10
(X_train, y_train), (X_test, y_test) = cifar10.load_data()
X_train.shape
plt.imshow(X_train[1])
X_train_sc = X_train.astype('float32') / 255.0
X_test_sc = X_test.astype('float32') / 255.0
y_train.shape
y_train_cat = to_categorical(y_train, 10)
y_test_cat = to_categorical(y_test, 10)
y_train_cat.shape
model = Sequential()
model.add(Conv2D(32, (3, 3),
                 padding='same',
                 input_shape=(32, 32, 3),
                 activation='relu'))
model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3), padding='same', activation='relu'))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])
model.summary()
model.fit(X_train_sc, y_train_cat,
          batch_size=32,
          epochs=2,
          validation_data=(X_test_sc, y_test_cat),
          shuffle=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10689608454139006, 'fraction_numerical': 0.027367565370545997, 'mean_word_length': 4.77621283255086, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '308053', 'license': 'MIT', 'n_tokens_mistral': 2628, 'n_tokens_neox': 2409, 'n_words': 715, 'path': 'solutions_do_not_open/Lab_03_DL Keras Intro Convolutional Models_solution.ipynb', 'repo_name': 'Dataweekends/global_AI_conference_Jan_2018', 'size': '7381'}