<jupyter_start><jupyter_text>**TASK-2**  Moving to the directory where data is stored<jupyter_code>cd ../input/task-2<jupyter_output>/kaggle/input/task-2<jupyter_text>Importing the required libraries<jupyter_code>import pandas as pd
import numpy as np
import tensorflow.keras
from keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.layers import Dense,Conv2D,Flatten,Dropout,MaxPool2D,AvgPool2D,BatchNormalization,Activation,Input
from tensorflow.keras import regularizers
from tensorflow.keras.models import Sequential,Model
from tensorflow.keras.applications import InceptionV3<jupyter_output><empty_output><jupyter_text>Variable learning rate with lr=0.001, deacy= 0.001/200, momnetum=0.9<jupyter_code>lr_schedule = tensorflow.keras.optimizers.SGD(
    lr=0.001,
    decay= 0.001/200,
    momentum=0.9)<jupyter_output><empty_output><jupyter_text>Data flow pipeline with addition of some noise (brightness and zoom)<jupyter_code>datagen= ImageDataGenerator(
zoom_range=0.15,
brightness_range = [0.3,1.4],
validation_split=0.20)
train_it=datagen.flow_from_directory('train_task2',class_mode='categorical',batch_size=20,target_size=(28,28),shuffle=True,color_mode='grayscale',subset='training')
test_it= datagen.flow_from_directory('train_task2',class_mode='categorical',batch_size=20,target_size=(28,28),shuffle=True,color_mode='grayscale',subset='validation')<jupyter_output>Found 320 images belonging to 10 classes.
Found 80 images belonging to 10 classes.<jupyter_text>Model architecture with structure inspired from VGG with addition of dropout and batch normalization layers to avoid overfitting and reduce the training time.<jupyter_code>model=Sequential()
model.add(Conv2D(32, (5, 5), padding="same",input_shape=(28,28,1)))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Conv2D(32, (3, 3), padding="same"))
model.add(BatchNormalization())
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(0.2))
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(0.2))
model.add(Conv2D(128, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Conv2D(128, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(0.2))
model.add(Conv2D(256, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Conv2D(256, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(50))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Dropout(0.3))
model.add(Dense(10))
model.add(Activation("softmax"))<jupyter_output><empty_output><jupyter_text>Compiling the model<jupyter_code>model.compile(loss='categorical_crossentropy', metrics=['accuracy'],optimizer= lr_schedule)<jupyter_output><empty_output><jupyter_text>Saving the weights in the file with max val accuracy<jupyter_code>from keras.callbacks import ModelCheckpoint
filepath='../../working/mymodel2.h5'
checkpoint = ModelCheckpoint(filepath, monitor='val_accuracy', verbose=1, save_best_only=True, mode='max')
callbacks_list = [checkpoint]<jupyter_output><empty_output><jupyter_text>Training the model<jupyter_code>model.fit(train_it,steps_per_epoch=int(320/20),epochs=50,validation_data=test_it,validation_steps=int(80/20),callbacks=callbacks_list)<jupyter_output>Epoch 1/50
16/16 [==============================] - 8s 527ms/step - loss: 0.0734 - accuracy: 0.9906 - val_loss: 0.5086 - val_accuracy: 0.8375

Epoch 00001: val_accuracy did not improve from 0.88750
Epoch 2/50
16/16 [==============================] - 8s 505ms/step - loss: 0.0899 - accuracy: 0.9719 - val_loss: 0.4731 - val_accuracy: 0.8625

Epoch 00002: val_accuracy did not improve from 0.88750
Epoch 3/50
16/16 [==============================] - 8s 510ms/step - loss: 0.0858 - accuracy: 0.9844 - val_loss: 0.7216 - val_accuracy: 0.7875

Epoch 00003: val_accuracy did not improve from 0.88750
Epoch 4/50
16/16 [==============================] - 8s 501ms/step - loss: 0.1421 - accuracy: 0.9594 - val_loss: 0.5044 - val_accuracy: 0.8125

Epoch 00004: val_accuracy did not improve from 0.88750
Epoch 5/50
16/16 [==============================] - 8s 504ms/step - loss: 0.0805 - accuracy: 0.9844 - val_loss: 0.5555 - val_accuracy: 0.8000

Epoch 00005: val_accuracy did not improve from 0.88750
Epoch 6/50[...]<jupyter_text>Max val accuracy reached to 90%  MNIST Data<jupyter_code>from keras.datasets import mnist
(train_X, train_y), (test_X, test_y) = mnist.load_data()
import tensorflow as tf
train_X=train_X.reshape(60000,28,28,1)
test_X= test_X.reshape(10000,28,28,1)<jupyter_output>Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 0s 0us/step<jupyter_text>Applying onehotencoding in the output data<jupyter_code>from sklearn.preprocessing import OneHotEncoder
test_y=test_y.reshape(10000,1)
# creating instance of one-hot-encoder
enc = OneHotEncoder(handle_unknown='ignore')
# passing bridge-types-cat column (label encoded values of bridge_types)
test_y = pd.DataFrame(enc.fit_transform(test_y).toarray())
# merge with main df bridge_df on key values

#bridge_df = bridge_df.join(enc_df)
from sklearn.preprocessing import OneHotEncoder
train_y=train_y.reshape(60000,1)
# creating instance of one-hot-encoder
enc = OneHotEncoder(handle_unknown='ignore')
# passing bridge-types-cat column (label encoded values of bridge_types)
train_y = pd.DataFrame(enc.fit_transform(train_y).toarray())<jupyter_output><empty_output><jupyter_text>Same model with random initialized weights<jupyter_code>model1=Sequential()
model1.add(Conv2D(32, (5, 5), padding="same",input_shape=(28,28,1)))
model1.add(Activation("relu"))
model1.add(BatchNormalization())
model1.add(Conv2D(32, (3, 3), padding="same"))
model1.add(BatchNormalization())
model1.add(MaxPool2D(pool_size=(2, 2)))
model1.add(Dropout(0.2))
model1.add(Conv2D(64, (3, 3), padding="same"))
model1.add(Activation("relu"))
model1.add(BatchNormalization())
model1.add(Conv2D(64, (3, 3), padding="same"))
model1.add(Activation("relu"))
model1.add(BatchNormalization())
model1.add(MaxPool2D(pool_size=(2, 2)))
model1.add(Dropout(0.2))
model1.add(Conv2D(128, (3, 3), padding="same"))
model1.add(Activation("relu"))
model1.add(BatchNormalization())
model1.add(Conv2D(128, (3, 3), padding="same"))
model1.add(Activation("relu"))
model1.add(BatchNormalization())
model1.add(MaxPool2D(pool_size=(2, 2)))
model1.add(Dropout(0.2))
model1.add(Conv2D(256, (3, 3), padding="same"))
model1.add(Activation("relu"))
model1.add(BatchNormalization())
model1.add(Conv2D(256, (3, 3), padding="same"))
model1.add(Activation("relu"))
model1.add(BatchNormalization())
model1.add(MaxPool2D(pool_size=(2, 2)))
model1.add(Dropout(0.2))
model1.add(Flatten())
model1.add(Dense(50))
model1.add(Activation("relu"))
model1.add(BatchNormalization())
model1.add(Dropout(0.3))
model1.add(Dense(10))
model1.add(Activation("softmax"))<jupyter_output><empty_output><jupyter_text>Compiling and training the model<jupyter_code>model1.compile(loss='categorical_crossentropy', metrics=['accuracy'],optimizer= lr_schedule)
model1.fit(train_X,train_y,batch_size=50,epochs=50,validation_data=(test_X,test_y),callbacks=callbacks_list)<jupyter_output>Epoch 1/50
1200/1200 [==============================] - 318s 264ms/step - loss: 0.7408 - accuracy: 0.7812 - val_loss: 0.0531 - val_accuracy: 0.9827
Epoch 2/50
1200/1200 [==============================] - 317s 264ms/step - loss: 0.1190 - accuracy: 0.9676 - val_loss: 0.0391 - val_accuracy: 0.9869
Epoch 3/50
1200/1200 [==============================] - 315s 263ms/step - loss: 0.0804 - accuracy: 0.9781 - val_loss: 0.0318 - val_accuracy: 0.9893
Epoch 4/50
1200/1200 [==============================] - 315s 262ms/step - loss: 0.0672 - accuracy: 0.9815 - val_loss: 0.0292 - val_accuracy: 0.9904
Epoch 5/50
1200/1200 [==============================] - 314s 261ms/step - loss: 0.0552 - accuracy: 0.9852 - val_loss: 0.0256 - val_accuracy: 0.9905
Epoch 6/50
1200/1200 [==============================] - 314s 262ms/step - loss: 0.0486 - accuracy: 0.9866 - val_loss: 0.0229 - val_accuracy: 0.9926
Epoch 7/50
1200/1200 [==============================] - 313s 261ms/step - loss: 0.0439 - accuracy: 0.9874 - val_[...]<jupyter_text>Compiling and training the same model with weights pretrained from dataset given in task-1 (The first trained model of this notebook is used below). Since the val accuracy reached above 99%, I stopped the training after 15 epochs due to which keyboard interrupt error is shown below(due to time limitations of the notebook session on kaggle)<jupyter_code>model.compile(loss='categorical_crossentropy', metrics=['accuracy'],optimizer= lr_schedule)
model.fit(train_X,train_y,batch_size=50,epochs=50,validation_data=(test_X,test_y))<jupyter_output>Epoch 1/50
1200/1200 [==============================] - 317s 263ms/step - loss: 0.6923 - accuracy: 0.8046 - val_loss: 0.0628 - val_accuracy: 0.9796
Epoch 2/50
1200/1200 [==============================] - 316s 263ms/step - loss: 0.1374 - accuracy: 0.9640 - val_loss: 0.0445 - val_accuracy: 0.9852
Epoch 3/50
1200/1200 [==============================] - 315s 263ms/step - loss: 0.0990 - accuracy: 0.9740 - val_loss: 0.0343 - val_accuracy: 0.9885
Epoch 4/50
1200/1200 [==============================] - 316s 263ms/step - loss: 0.0774 - accuracy: 0.9792 - val_loss: 0.0289 - val_accuracy: 0.9901
Epoch 5/50
1200/1200 [==============================] - 316s 264ms/step - loss: 0.0643 - accuracy: 0.9832 - val_loss: 0.0285 - val_accuracy: 0.9904
Epoch 6/50
1200/1200 [==============================] - 317s 264ms/step - loss: 0.0584 - accuracy: 0.9840 - val_loss: 0.0243 - val_accuracy: 0.9906
Epoch 7/50
1200/1200 [==============================] - 317s 264ms/step - loss: 0.0531 - accuracy: 0.9857 - val_[...]
{'fraction_non_alphanumeric': 0.19811967486044463, 'fraction_numerical': 0.10253648026637939, 'mean_word_length': 7.059984214680347, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '159497', 'license': 'MIT', 'n_tokens_mistral': 4343, 'n_tokens_neox': 3656, 'n_words': 700, 'path': 'task-2/task-2_wt/midas.ipynb', 'repo_name': 'Saanidhyavats/MIDAS_Project', 'size': '10211'}