<jupyter_start><jupyter_text>*Data Science Unit 4 Sprint 3 Assignment 2* Convolutional Neural Networks (CNNs)  Assignment- Part 1: Pre-Trained Model- Part 2: Custom CNN Model- Part 3: CNN with Data AugmentationYou will apply three different CNN models to a binary image classification model using Keras. Classify images of Mountains (`./data/mountain/*`) and images of forests (`./data/forest/*`). Treat mountains as the postive class (1) and the forest images as the negative (zero). |Mountain (+)|Forest (-)||---|---||![](https://github.com/blakelobato/DS-Unit-4-Sprint-3-Deep-Learning/blob/master/module2-convolutional-neural-networks/data/mountain/art1131.jpg?raw=1)|![](https://github.com/blakelobato/DS-Unit-4-Sprint-3-Deep-Learning/blob/master/module2-convolutional-neural-networks/data/forest/cdmc317.jpg?raw=1)|The problem is realively difficult given that the sample is tiny: there are about 350 observations per class. This sample size might be something that you can expect with prototyping an image classification problem/solution at work. Get accustomed to evaluating several differnet possible models.  Pre - Trained ModelLoad a pretrained network from Keras, [ResNet50](https://tfhub.dev/google/imagenet/resnet_v1_50/classification/1) - a 50 layer deep network trained to recognize [1000 objects](https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt). Starting usage:```pythonimport numpy as npfrom tensorflow.keras.applications.resnet50 import ResNet50from tensorflow.keras.preprocessing import imagefrom tensorflow.keras.applications.resnet50 import preprocess_input, decode_predictionsfrom tensorflow.keras.layers import Dense, GlobalAveragePooling2D()from tensorflow.keras.models import Model  This is the functional APIresnet = ResNet50(weights='imagenet', include_top=False)```The `include_top` parameter in `ResNet50` will remove the full connected layers from the ResNet model. The next step is to turn off the training of the ResNet layers. We want to use the learned parameters without updating them in future training passes. ```pythonfor layer in resnet.layers:    layer.trainable = False```Using the Keras functional API, we will need to additional additional full connected layers to our model. We we removed the top layers, we removed all preivous fully connected layers. In other words, we kept only the feature processing portions of our network. You can expert with additional layers beyond what's listed here. The `GlobalAveragePooling2D` layer functions as a really fancy flatten function by taking the average of each of the last convolutional layer outputs (which is two dimensional still). ```pythonx = res.outputx = GlobalAveragePooling2D()(x)  This layer is a really fancy flattenx = Dense(1024, activation='relu')(x)predictions = Dense(1, activation='sigmoid')(x)model = Model(res.input, predictions)```Your assignment is to apply the transfer learning above to classify images of Mountains (`./data/mountain/*`) and images of forests (`./data/forest/*`). Treat mountains as the postive class (1) and the forest images as the negative (zero). Steps to complete assignment: 1. Load in Image Data into numpy arrays (`X`) 2. Create a `y` for the labels3. Train your model with pretrained layers from resnet4. Report your model's accuracy<jupyter_code>from google.colab import drive
drive.mount('/content/drive')
import os
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import backend as K
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Flatten, Dense, Dropout, GlobalAveragePooling2D
from tensorflow.keras.applications.inception_resnet_v2 import InceptionResNetV2, preprocess_input
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau


DATASET_PATH  = '/content/drive/My Drive/Colab Notebooks/Unit 4 Sprint 3 Major NN Arch/data'
from skimage.io import imread_collection
from skimage.transform import resize
from sklearn.model_selection import train_test_split<jupyter_output><empty_output><jupyter_text>Load in Data![skimage-logo](https://scikit-image.org/_static/img/logo.png)Check out out [`skimage`](https://scikit-image.org/) for useful functions related to processing the images. In particular checkout the documentation for `skimage.io.imread_collection` and `skimage.transform.resize`.<jupyter_code>#your path 
forests = '/content/drive/My Drive/Colab Notebooks/Unit 4 Sprint 3 Major NN Arch/data/train/forest/*.jpg'
mountains = '/content/drive/My Drive/Colab Notebooks/Unit 4 Sprint 3 Major NN Arch/data/train/mountain/*.jpg'
val_forests = '/content/drive/My Drive/Colab Notebooks/Unit 4 Sprint 3 Major NN Arch/data/validation/forest/*.jpg'
val_mountains = '/content/drive/My Drive/Colab Notebooks/Unit 4 Sprint 3 Major NN Arch/data/validation/mountain/*.jpg'
#creating a collection with the available images
forests = imread_collection(forests).concatenate()
mountains = imread_collection(mountains).concatenate()
forests.shape
y_0 =  np.zeros(forests.shape[0])
y_1 =  np.ones(mountains.shape[0])
X = np.concatenate([forests, mountains])
X = resize(X, (520,256,256,3))
y = np.concatenate([y_0, y_1])
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=.5)
x_train<jupyter_output><empty_output><jupyter_text>Instatiate Model<jupyter_code>x_train.shape
# Create the base model from the pre-trained model ResNet
base_model = tf.keras.applications.resnet50.ResNet50(weights='imagenet', include_top=False, input_shape= (256,256,3))

for layer in base_model.layers:
    layer.trainable = False
base_model.trainable = False
# Let's take a look at the base model architecture
base_model.summary()
DATASET_PATH  = '/content/drive/My Drive/Colab Notebooks/Unit 4 Sprint 3 Major NN Arch/data'
IMAGE_SIZE    = (255, 255)
NUM_CLASSES   = 2
BATCH_SIZE    = 5  # try reducing batch size or freeze more layers if your GPU runs out of memory
train_data = ImageDataGenerator(preprocessing_function=preprocess_input)

train_batches = train_data.flow_from_directory(DATASET_PATH + '/train',
                                                  target_size=IMAGE_SIZE,
                                                  class_mode='categorical',
                                                  shuffle=True,
                                                  batch_size=BATCH_SIZE)

valid_data = ImageDataGenerator(preprocessing_function=preprocess_input)



valid_batches = valid_data.flow_from_directory(DATASET_PATH + '/validation',
                                                  target_size=IMAGE_SIZE,
                                                  class_mode='categorical',
                                                  shuffle=True,
                                                  batch_size=BATCH_SIZE)
# show class indices
print('****************')
for cls, idx in train_batches.class_indices.items():
    print('Class #{} = {}'.format(idx, cls))
print('****************')
base_model = tf.keras.applications.resnet50.ResNet50(weights='imagenet', include_top=False, input_shape=(IMAGE_SIZE[0],IMAGE_SIZE[1],3))

for layer in base_model.layers:
    layer.trainable = False
x = base_model.output
x = Flatten()(x)
output_layer = Dense(NUM_CLASSES, activation='softmax', name='softmax')(x)
network = Model(inputs=base_model.input, outputs=output_layer)
network.compile(optimizer=Adam(lr=0.001),
                  loss='categorical_crossentropy', metrics=['accuracy'])
stop = EarlyStopping(monitor='val_accuracy', mode='max', min_delta=0.01, patience=3, verbose=1)
network.fit(
        train_batches,
        epochs=10,
        validation_data=valid_batches,
        verbose=1,
        steps_per_epoch = train_batches.samples // BATCH_SIZE,
        callbacks=[stop],
        validation_steps = valid_batches.samples // BATCH_SIZE)<jupyter_output>Epoch 1/10
104/104 [==============================] - 140s 1s/step - loss: 3.0188 - accuracy: 0.7538 - val_loss: 2.8799 - val_accuracy: 0.7000
Epoch 2/10
104/104 [==============================] - 137s 1s/step - loss: 1.3443 - accuracy: 0.8500 - val_loss: 0.6967 - val_accuracy: 0.9167
Epoch 3/10
104/104 [==============================] - 135s 1s/step - loss: 0.8530 - accuracy: 0.9115 - val_loss: 3.7993 - val_accuracy: 0.7222
Epoch 4/10
104/104 [==============================] - 136s 1s/step - loss: 0.1349 - accuracy: 0.9827 - val_loss: 0.6439 - val_accuracy: 0.8944
Epoch 5/10
104/104 [==============================] - 136s 1s/step - loss: 0.0662 - accuracy: 0.9865 - val_loss: 1.4625 - val_accuracy: 0.8278
Epoch 00005: early stopping<jupyter_text>Custom CNN ModelIn this step, write and train your own convolutional neural network using Keras. You can use any architecture that suits you as long as it has at least one convolutional and one pooling layer at the beginning of the network - you can add more if you want.<jupyter_code>x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=.5)
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Conv2D, MaxPooling2D, Flatten

# Setup Architecture
model = Sequential()
model.add(Conv2D(32, (3,3), activation='relu', input_shape=(256,256,3)))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3,3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(1, activation='softmax'))

# Compile Model
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.summary()
# Print summary
model.fit(train_batches,
          epochs=10,
          validation_data=valid_batches)
# Normalize from 0-1
x_train = x_train.astype('float32') / 260.
x_test = x_test.astype('float32') / 260.

# Reshaping the array to 4-dims so that it can work with the Tensorflow.Keras API
#X_train = X_train.reshape(X_train.shape[0], 28, 28, 1)
#X_test = X_test.reshape(X_test.shape[0], 28, 28, 1)

# Setup Architecture
model = Sequential()
model.add(Conv2D(32, (3,3), activation='relu', input_shape=(256,256,3)))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3,3), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(10, activation='softmax'))

# Compile Model
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Print summary
model.fit(x_train, y_train, 
          epochs=5, 
          validation_data=(x_test, y_test))<jupyter_output>Epoch 1/5
9/9 [==============================] - 28s 3s/step - loss: 1.3718 - accuracy: 0.4769 - val_loss: 0.8538 - val_accuracy: 0.5231
Epoch 2/5
9/9 [==============================] - 28s 3s/step - loss: 0.7843 - accuracy: 0.5154 - val_loss: 0.7391 - val_accuracy: 0.5231
Epoch 3/5
9/9 [==============================] - 28s 3s/step - loss: 0.7570 - accuracy: 0.5000 - val_loss: 0.7241 - val_accuracy: 0.4769
Epoch 4/5
9/9 [==============================] - 28s 3s/step - loss: 0.7040 - accuracy: 0.5231 - val_loss: 0.7331 - val_accuracy: 0.5231
Epoch 5/5
9/9 [==============================] - 28s 3s/step - loss: 0.7128 - accuracy: 0.5231 - val_loss: 0.7279 - val_accuracy: 0.4769<jupyter_text>Custom CNN Model with Image Manipulations *This a stretch goal, and it's relatively difficult*To simulate an increase in a sample of image, you can apply image manipulation techniques: cropping, rotation, stretching, etc. Luckily Keras has some handy functions for us to apply these techniques to our mountain and forest example. Check out these resources to help you get started: 1. [Keras `ImageGenerator` Class](https://keras.io/preprocessing/image/imagedatagenerator-class)2. [Building a powerful image classifier with very little data](https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html)<jupyter_code># State Code for Image Manipulation Here<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1317013463892289, 'fraction_numerical': 0.04332925336597307, 'mean_word_length': 4.423008849557522, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 17, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '29517', 'license': 'MIT', 'n_tokens_mistral': 4123, 'n_tokens_neox': 3710, 'n_words': 1055, 'path': 'module2-convolutional-neural-networks/432A_CNN.ipynb', 'repo_name': 'blakelobato/DS-Unit-4-Sprint-3-Deep-Learning', 'size': '12255'}