<jupyter_start><jupyter_text>Advanced Model with BathNormalization & Adam using keras The **conv-block-model** looks like: (the shape will be different)              The whole **model** looks like:              The **detail model** :<jupyter_code>import h5py
import numpy as np
from keras import layers
import keras.backend as K
from keras.models import Model
import matplotlib.pyplot as plt
from keras.callbacks import Callback
from keras.layers import Input, Dense, Activation, BatchNormalization, Flatten, Conv2D, AveragePooling2D, MaxPooling2D

K.set_image_data_format('channels_last')
%matplotlib tk<jupyter_output>Using TensorFlow backend.<jupyter_text>1. Build the function of defining model<jupyter_code>def AdvancedModel(X_shape):
    
    X_input = Input(shape=X_shape, name='X_input')
    
    # 1st CONV-BLOCK
    # Conv2D: strides default (1,1), padding default 'valid'
    # Not use bias but center(offset, or named: 'beta')
    Z1 = Conv2D(filters=8, kernel_size=(3,3), use_bias=False, kernel_initializer='glorot_uniform', name='conv_1')(X_input) 
    Z2 = BatchNormalization(scale=False, moving_variance_initializer='glorot_uniform', name='bn_2')(Z1)
    A = Activation('relu', name='relu_3')(Z2)
    X = AveragePooling2D(pool_size=(3,3), strides=(1,1), name='avg_pool_4')(A)
    
    # 2nd CONV-BLOCK
    Z1 = Conv2D(filters=12, kernel_size=(5,5), use_bias=False, kernel_initializer='glorot_uniform', name='conv_5')(X) 
    Z2 = BatchNormalization(scale=False, moving_variance_initializer='glorot_uniform', name='bn_6')(Z1)
    A = Activation('relu', name='relu_7')(Z2)
    X = AveragePooling2D(pool_size=(5,5), strides=(1,1), name='avg_pool_8')(A)
    
    # 3rd CONV-BLOCK
    Z1 = Conv2D(filters=16, kernel_size=(5,5), use_bias=False, kernel_initializer='glorot_uniform', name='conv_9')(X) 
    Z2 = BatchNormalization(scale=False, moving_variance_initializer='glorot_uniform', name='bn_10')(Z1)
    A = Activation('relu', name='relu_11')(Z2)
    X = AveragePooling2D(pool_size=(5,5), strides=(1,1), name='avg_pool_12')(A)
    
    # 4th CONV-BLOCK
    Z1 = Conv2D(filters=20, kernel_size=(5,5), use_bias=False, kernel_initializer='glorot_uniform', name='conv_13')(X) 
    Z2 = BatchNormalization(scale=False, moving_variance_initializer='glorot_uniform', name='bn_14')(Z1)
    A = Activation('relu', name='relu_15')(Z2)
    X = AveragePooling2D(pool_size=(5,5), strides=(1,1), name='avg_pool_16')(A)
    
    # flatten
    X = Flatten()(X)
    
    # 5th NN-BLOCK
    Z1 = Dense(units=128, use_bias=False, kernel_initializer='glorot_uniform', name='fc_17')(X)
    Z2 = BatchNormalization(scale=False, moving_variance_initializer='glorot_uniform', name='bn_18')(Z1)
    X = Activation('relu', name='relu_19')(Z2)
    
    # 6th NN-BLOCK
    Z1 = Dense(units=64, use_bias=False, kernel_initializer='glorot_uniform', name='fc_20')(X)
    Z2 = BatchNormalization(scale=False, moving_variance_initializer='glorot_uniform', name='bn_21')(Z1)
    X = Activation('relu', name='relu_22')(Z2)
    
    # 7th NN-BLOCK
    Z1 = Dense(units=32, use_bias=False, kernel_initializer='glorot_uniform', name='fc_23')(X)
    Z2 = BatchNormalization(scale=False, moving_variance_initializer='glorot_uniform', name='bn_24')(Z1)
    X = Activation('relu', name='relu_25')(Z2)
    
    # 8th NN-BLOCK
    Y = Dense(units=1, activation='sigmoid', kernel_initializer='glorot_uniform', name='fc_26')(X)
    
    model = Model(inputs = X_input, outputs = Y, name='AdvancedModel')
    
    return model<jupyter_output><empty_output><jupyter_text>2. Create a callback to record the losses during training.<jupyter_code>class LossRecorder(Callback):
    def on_train_begin(self, logs={}):
        self.losses = []

    def on_batch_end(self, batch, logs={}):
        self.losses.append(logs.get('loss'))<jupyter_output><empty_output><jupyter_text>3. Create a model<jupyter_code>model = AdvancedModel(X_shape=(32,32,3))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>4. Load the dataset<jupyter_code>with h5py.File('datasets/car-v2.h5', 'r', driver='core') as f:
    DIVIDE = 6400
    X_test, Y_test = f['X'][DIVIDE:], f['Y'][DIVIDE:].reshape(-1, 1)
    X_train, Y_train = f['X'][:DIVIDE], f['Y'][:DIVIDE].reshape(-1, 1)
print('X_shape: ', X_train.shape)
print('Y_shape: ', Y_train.shape)
loss_recorder = LossRecorder()<jupyter_output><empty_output><jupyter_text>5. OK, it's time to train the model!<jupyter_code>model.fit(X_train, Y_train, batch_size = 32, epochs = 10, callbacks=[loss_recorder])<jupyter_output>Epoch 1/10
6400/6400 [==============================] - 16s - loss: 0.4034 - acc: 0.8342    
Epoch 2/10
6400/6400 [==============================] - 16s - loss: 0.2913 - acc: 0.8842    
Epoch 3/10
6400/6400 [==============================] - 16s - loss: 0.2583 - acc: 0.8958    
Epoch 4/10
6400/6400 [==============================] - 16s - loss: 0.2334 - acc: 0.9077    
Epoch 5/10
6400/6400 [==============================] - 16s - loss: 0.2219 - acc: 0.9133    
Epoch 6/10
6400/6400 [==============================] - 16s - loss: 0.2159 - acc: 0.9163    
Epoch 7/10
6400/6400 [==============================] - 16s - loss: 0.2045 - acc: 0.9186    
Epoch 8/10
6400/6400 [==============================] - 16s - loss: 0.1962 - acc: 0.9209    
Epoch 9/10
6400/6400 [==============================] - 16s - loss: 0.1962 - acc: 0.9216    
Epoch 10/10
6400/6400 [==============================] - 16s - loss: 0.1932 - acc: 0.9234<jupyter_text>6. Evaluate the performance of model<jupyter_code>preds = model.evaluate(X_test, Y_test)
print ("Final Loss \t = " + str(preds[0]))
print ("Test Accuracy \t = " + str(preds[1]))<jupyter_output>800/800 [==============================] - 0s     
Final Loss 	 = 0.235009089112
Test Accuracy 	 = 0.91625<jupyter_text>The loss and accuracy are both little worse than training. But it's still ok.   7. Now, let's show the loss.<jupyter_code>num = len(loss_recorder.losses)
plt.plot(range(num), loss_recorder.losses)
plt.xlabel('number_of_batch')
plt.ylabel('loss')
plt.show()<jupyter_output><empty_output><jupyter_text>8. Check the model archtecture.<jupyter_code>model.summary()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
X_input (InputLayer)         (None, 32, 32, 3)         0         
_________________________________________________________________
conv_1 (Conv2D)              (None, 30, 30, 8)         216       
_________________________________________________________________
bn_2 (BatchNormalization)    (None, 30, 30, 8)         24        
_________________________________________________________________
relu_3 (Activation)          (None, 30, 30, 8)         0         
_________________________________________________________________
avg_pool_4 (AveragePooling2D (None, 28, 28, 8)         0         
_________________________________________________________________
conv_5 (Conv2D)              (None, 24, 24, 12)        2400      
_________________________________________________________________
bn_6 (Batc[...]
{'fraction_non_alphanumeric': 0.18586278586278587, 'fraction_numerical': 0.06541926541926542, 'mean_word_length': 4.857142857142857, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '44060', 'license': 'Apache-2.0', 'n_tokens_mistral': 2791, 'n_tokens_neox': 2476, 'n_words': 473, 'path': 'car_project/car_detection_v2.ipynb', 'repo_name': 'LonelySqrt/CVProjects', 'size': '7215'}