<jupyter_start><jupyter_text>Deep CNNHere we can:* Retrain the entire network with randomly initialized weights* Train network with pre-trained encoder weights and randomly initialized FC layer weights(fine-tuning)Reconstruction of images plotted at the end of notebook.<jupyter_code>from tensorflow.keras.layers import Input, Dense, Conv2D, MaxPooling2D, UpSampling2D, Dropout, Flatten
from tensorflow.keras.models import Model, Sequential
from tensorflow.keras import backend as K
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from math import ceil
from random import randint
from shared.autoencoderHelpers import read_n_images, generate_img_from_folder, get_input_shape, get_num_examples, plot_history, get_images, bgr2rgb, plot_reconstruction<jupyter_output><empty_output><jupyter_text>Generate model(s)* Encoder* FFNN **Set the layers for auto-encoder**<jupyter_code>DATA_DIR = '../data'
autoencoder = tf.keras.models.load_model('../models/autoencoder/twoClassTest/model') 
autoencoder.summary()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_layer (InputLayer)     (None, 200, 200, 3)       0         
_________________________________________________________________
enc_conv1 (Conv2D)           (None, 200, 200, 16)      448       
_________________________________________________________________
enc_max_pool1 (MaxPooling2D) (None, 100, 100, 16)      0         
_________________________________________________________________
enc_conv2 (Conv2D)           (None, 100, 100, 8)       1160      
_________________________________________________________________
enc_max_pool2 (MaxPooling2D) (None, 50, 50, 8)         0         
_________________________________________________________________
enc_conv3 (Conv2D)           (None, 50, 50, 8)         584       
_________________________________________________________________
enc_max_po[...]<jupyter_text>Visual representationVisual representation of what we are doing in Keras essentially.We train the models in 3 distinct ways:* Use model as a whole (CNN, green)    * Initialize the weights randomly and train the whole model    * Initialize the encoder layer weights as they are and train the whole model (fine tuning)* Use model as an encoder (red) and classifier (blue)    * Keep the encoder layer with the trained weights and initialize the classifier model weights randomly and train **only** the classifier model.<jupyter_code>num_classes = get_num_classes(DATA_DIR)

layer_names = [autoencoder.layers[i].name for i in range(7)]
print("Layers of encoder:\n{}".format(layer_names))

encoder = Model(inputs=autoencoder.inputs, outputs=autoencoder.layers[6].output)

for layer in encoder.layers:
    layer.trainable = True  # Set to true to allow fine-tuning

encoder_out = autoencoder.layers[6].output
x = Flatten()(encoder.output)
x = Dense(500,activation='relu')(x)
x = Dropout(rate=0.5)(x)
x = Dense(500,activation='relu')(x)
x = Dropout(rate=0.5)(x)
output_full_model = Dense(num_classes, activation=tf.nn.sigmoid, name='output_layer')(x) # Maybe sigmoid


clf_frozen = Model(inputs=encoder.inputs, outputs=output_full_model)
[layer.name for layer in autoencoder.layers]
encoder.summary()
clf_frozen.summary()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_layer (InputLayer)     (None, 200, 200, 3)       0         
_________________________________________________________________
enc_conv1 (Conv2D)           (None, 200, 200, 16)      448       
_________________________________________________________________
enc_max_pool1 (MaxPooling2D) (None, 100, 100, 16)      0         
_________________________________________________________________
enc_conv2 (Conv2D)           (None, 100, 100, 8)       1160      
_________________________________________________________________
enc_max_pool2 (MaxPooling2D) (None, 50, 50, 8)         0         
_________________________________________________________________
enc_conv3 (Conv2D)           (None, 50, 50, 8)         584       
_________________________________________________________________
enc_max_po[...]<jupyter_text>Compile the model and get model summariesCompiling only happens for `clf_model` model (as we are not training the encoder).<jupyter_code>clf_frozen.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>Training the frozen classifier* Tune for batch size, number of epochs, etc.<jupyter_code>batchSize = 8
epochs = 30

BATCH_SIZE_TRAIN = 10
NUM_SAMPLES_TRAIN = get_num_examples(DATA_DIR, 'training')
STEPS_PER_EPOCH = ceil(NUM_SAMPLES_TRAIN/BATCH_SIZE_TRAIN)

BATCH_SIZE_VAL= 10
NUM_SAMPLES_VAL = get_num_examples(DATA_DIR, 'validation')
VALIDATION_STEPS=ceil(NUM_SAMPLES_VAL/BATCH_SIZE_VAL)

EPOCHS = 30

early_stop = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
history = clf_frozen.fit_generator(generate_img_label_from_folder(DATA_DIR, 'training', BATCH_SIZE_TRAIN), shuffle=True,
                                    validation_data=generate_img_label_from_folder(DATA_DIR, 'validation', BATCH_SIZE_VAL),
                                    steps_per_epoch=STEPS_PER_EPOCH, validation_steps=VALIDATION_STEPS,
                                    epochs=EPOCHS)<jupyter_output>Epoch 1/30
18/18 [==============================] - 2s 126ms/step - loss: 0.6257 - acc: 0.6705
52/52 [==============================] - 16s 312ms/step - loss: 0.6950 - acc: 0.5846 - val_loss: 0.6257 - val_acc: 0.6705
Epoch 2/30
18/18 [==============================] - 2s 116ms/step - loss: 0.5901 - acc: 0.7225
52/52 [==============================] - 16s 302ms/step - loss: 0.6251 - acc: 0.6788 - val_loss: 0.5901 - val_acc: 0.7225
Epoch 3/30
18/18 [==============================] - 3s 143ms/step - loss: 0.4957 - acc: 0.7977
52/52 [==============================] - 17s 328ms/step - loss: 0.5740 - acc: 0.7673 - val_loss: 0.4957 - val_acc: 0.7977
Epoch 4/30
18/18 [==============================] - 2s 137ms/step - loss: 0.4537 - acc: 0.8324
52/52 [==============================] - 18s 338ms/step - loss: 0.4572 - acc: 0.8135 - val_loss: 0.4537 - val_acc: 0.8324
Epoch 5/30
18/18 [==============================] - 2s 111ms/step - loss: 0.5068 - acc: 0.8266
52/52 [==============================[...]<jupyter_text>Accessment of model perfomance* Visualize the training history* Generate predictions for auto-encoder (reconstructed images)* Visualize original and reconstructed images (for comparison)<jupyter_code>plot_history(history)
#Decoding images
BATCH_SIZE_TEST = 10
NUM_SAMPLES_TEST = get_num_examples(DATA_DIR, 'test')
STEPS_TEST = ceil(NUM_SAMPLES_TEST/BATCH_SIZE_TEST)

evaluation = clf_frozen.evaluate_generator(generate_img_label_from_folder(DATA_DIR, 'test', BATCH_SIZE_TEST),steps=STEPS_TEST)
print("The test set after {} epochs:\nLoss: {}\nAccuracy: {}".format(EPOCHS, evaluation[0], evaluation[-1]))<jupyter_output>The test set after 2 epochs:
Loss: 4.8704644883120505
Accuracy: 0.6960893869400024<jupyter_text>Melted classifier<jupyter_code>in_shape = get_input_shape(DATA_DIR, 'training')
clf_melted = Sequential([
    Conv2D(16, (3,3), activation='relu', padding='same', name='enc_conv1', input_shape=in_shape),
    MaxPooling2D((2, 2), padding='same', name='enc_max_pool1'),
    Conv2D(8, (3, 3), activation='relu', padding='same', name='enc_conv2'),
    MaxPooling2D((2, 2), padding='same', name='enc_max_pool2'),
    Conv2D(8, (3, 3), activation='relu', padding='same', name='enc_conv3'),
    MaxPooling2D((2, 2), padding='same', name='enc_max_pool3'),
    Flatten(),
    Dense(2048, activation=tf.nn.relu, name='fc_1'),
    Dense(1024, activation=tf.nn.relu, name='fc_2'),
    Dropout(rate=0.4, name='dropout'),
    Dense(num_classes, activation=tf.nn.softmax, name='output_layer') # Maybe sigmoid
])
clf_melted.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
# Same as before, but all parameters are trainable
clf_melted.summary()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
enc_conv1 (Conv2D)           (None, 200, 200, 16)      448       
_________________________________________________________________
enc_max_pool1 (MaxPooling2D) (None, 100, 100, 16)      0         
_________________________________________________________________
enc_conv2 (Conv2D)           (None, 100, 100, 8)       1160      
_________________________________________________________________
enc_max_pool2 (MaxPooling2D) (None, 50, 50, 8)         0         
_________________________________________________________________
enc_conv3 (Conv2D)           (None, 50, 50, 8)         584       
_________________________________________________________________
enc_max_pool3 (MaxPooling2D) (None, 25, 25, 8)         0         
_________________________________________________________________
flatten_1 [...]<jupyter_text>Training the classifier* Tune for batch size, number of epochs, etc.<jupyter_code>batchSize = 8
epochs = 30

BATCH_SIZE_TRAIN = 10
NUM_SAMPLES_TRAIN = get_num_examples(DATA_DIR, 'training')
STEPS_PER_EPOCH = ceil(NUM_SAMPLES_TRAIN/BATCH_SIZE_TRAIN)

BATCH_SIZE_VAL= 10
NUM_SAMPLES_VAL = get_num_examples(DATA_DIR, 'validation')
VALIDATION_STEPS=ceil(NUM_SAMPLES_VAL/BATCH_SIZE_VAL)

EPOCHS = 2

early_stop = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
history = clf_melted.fit_generator(generate_img_label_from_folder(DATA_DIR, 'training', BATCH_SIZE_TRAIN), shuffle=True,
                                    validation_data=generate_img_label_from_folder(DATA_DIR, 'validation', BATCH_SIZE_VAL),
                                    steps_per_epoch=STEPS_PER_EPOCH, validation_steps=VALIDATION_STEPS,
                                    epochs=EPOCHS)<jupyter_output>Epoch 1/2
54/54 [==============================] - 11s 195ms/step - loss: 0.4744 - acc: 0.7974
162/162 [==============================] - 97s 601ms/step - loss: 0.5470 - acc: 0.7904 - val_loss: 0.4744 - val_acc: 0.7974
Epoch 2/2
54/54 [==============================] - 10s 190ms/step - loss: 0.6494 - acc: 0.7359
162/162 [==============================] - 95s 589ms/step - loss: 0.4544 - acc: 0.8036 - val_loss: 0.6494 - val_acc: 0.7359<jupyter_text>Accessment of model perfomance* Visualize the training history* Generate predictions for auto-encoder (reconstructed images)* Visualize original and reconstructed images (for comparison)<jupyter_code>plot_history(history)
#Decoding images
BATCH_SIZE_TEST = 10
NUM_SAMPLES_TEST = get_num_examples(DATA_DIR, 'test')
STEPS_TEST = ceil(NUM_SAMPLES_TEST/BATCH_SIZE_TEST)

evaluation = clf_melted.evaluate_generator(generate_img_label_from_folder(DATA_DIR, 'test', BATCH_SIZE_TEST),steps=STEPS_TEST)
print("The test set after {} epochs:\nLoss: {}\nAccuracy: {}".format(EPOCHS, evaluation[0], evaluation[-1]))<jupyter_output>The test set after 2 epochs:
Loss: 0.6628557173190294
Accuracy: 0.7281191945075989
{'fraction_non_alphanumeric': 0.14746584398413398, 'fraction_numerical': 0.05958572058175408, 'mean_word_length': 5.047974413646055, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 30, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '415740', 'license': 'MIT', 'n_tokens_mistral': 3899, 'n_tokens_neox': 3346, 'n_words': 768, 'path': 'scripts/02.1_classifier_prototype.ipynb', 'repo_name': 'RushilDaya/DeepVision', 'size': '11345'}