<jupyter_start><jupyter_text>Digit Recognizer (MNIST)=================================This is an implementation of [Kaggle competition for recognizing digits](https://www.kaggle.com/c/digit-recognizer)<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib notebook
import ImageUtils
import DataUtils<jupyter_output><empty_output><jupyter_text>Loading the datasetFirst let's investigate the training dataset, we first read the file, with seperater as `,` and specify that the first row is the header. We can then use the header as our dictionary key<jupyter_code>train_data = pd.read_csv('train.csv', sep=',',header=0)
training_labels = train_data['label'].values
training_values = train_data.drop('label', 1).values

evaluation_data = pd.read_csv('test.csv', sep=',',header=0)
evaluation_values = evaluation_data.values

print("There are a total of {} training images".format(len(train_data)))
print("There are a total of {} evaluation images".format(len(evaluation_data)))

training_values = training_values/255
evaluation_values = evaluation_values/255<jupyter_output>There are a total of 42000 training images
There are a total of 28000 evaluation images<jupyter_text>Dataset visualizationNow we try to investigate and print some of the data to understand the dataset, we start from index 0 and try every other image<jupyter_code>for x in range(2, 8, 2):
    row = DataUtils.reshape_row(training_values[x])
    ImageUtils.show_mnist_image(row)
    print("The image is {}".format(training_labels[x]))<jupyter_output>The image is 1
The image is 0
The image is 7<jupyter_text>Let's try visualizing the test dataset, we only have test dataset values, but no targets, which means we would have to split out training dataset into test dataset for training the neural network,The reason why the Kaggel provided test dataset exists, is so that we run the test dataset through our neural network and output the values which would be checked against the test targets (that data is only with Kaggle)<jupyter_code>for x in range(0, 3):
    row = DataUtils.reshape_row(evaluation_values[x])
    ImageUtils.show_mnist_image(row)<jupyter_output><empty_output><jupyter_text>One hot encoding the targetsThe output target is actual numerical values, so we would need to one hot encode the values<jupyter_code>from keras.utils import to_categorical

print("Before one-hot encoding:\n {}".format(training_labels[:10]))

y_train = to_categorical(training_labels)

print("\n\nAfter one-hot encoding:\n {}".format(y_train[:10]))<jupyter_output>Using TensorFlow backend.<jupyter_text>Reshape input images<jupyter_code>x_train = training_values.reshape(-1, 28, 28, 1)
print("Shape of training_values {}".format(x_train.shape))<jupyter_output>Shape of training_values (42000, 28, 28, 1)<jupyter_text>Creating our Convolution ModelWe would be using Keras Sequential layers to generate a model to fit the data<jupyter_code>from keras.models import Sequential
from keras.layers import Dense, Conv2D, MaxPooling2D, Dropout, Flatten, InputLayer
from keras.optimizers import SGD, Adamax
from keras.losses import categorical_crossentropy
from keras.callbacks import ModelCheckpoint, Callback

model = Sequential()

model.add(InputLayer(input_shape=(28, 28, 1)))
model.add(Conv2D(32, kernel_size=6, strides=1, activation='relu', padding='same'))
model.add(MaxPooling2D(2, 2,padding='same'))
model.add(Dropout(0.6))

model.add(Conv2D(64, kernel_size=6, strides=1, activation='relu', padding='same'))
model.add(MaxPooling2D(2, 2,padding='same'))
model.add(Dropout(0.7))

model.add(Flatten())

model.add(Dense(512, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))

model.compile(loss=categorical_crossentropy,
              optimizer=Adamax(lr=0.02),
              metrics=['accuracy'])
model.summary()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 28, 28, 1)         0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 28, 28, 32)        1184      
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 14, 14, 32)        0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 14, 14, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 14, 14, 64)        73792     
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 7, 7, 64)          0         
_________________________________________________________________
dropout_2 [...]<jupyter_text>Training on dataWe are splitting the provides training data into training and validation to guide the model towards accuracy. We have to do this because test.csv is not labelled and is only provided for us to predict the values for submission.<jupyter_code>class LossHistory(Callback):
    def on_train_begin(self, logs={}):
        self.losses = []

    def on_batch_end(self, batch, logs={}):
        self.losses.append(logs.get('loss'))
       

checkpoint = ModelCheckpoint("./best.hdf5", monitor='val_acc', save_best_only=True,  mode='max')
history = LossHistory()

model.fit(x_train, y_train, validation_split=0.2, epochs=3, batch_size=64, \
        callbacks=[checkpoint, history])
plt.figure(figsize=(15,10))
plt.plot(history.losses)<jupyter_output><empty_output><jupyter_text>Predict on dataNow that we have trained our model, we would take the evaluation data from `train.csv`, reshape it and pass it through the predict method or the model.Once we get the probability values of each data point, we just end up taking the maximum probability class for each provided data entry.<jupyter_code>x_evaluate = evaluation_values.reshape(-1, 28, 28, 1)
y_evaluate = model.predict(x_evaluate)
classes_evaluate = np.argmax(y_evaluate, axis=1)
print(classes_evaluate)<jupyter_output>[2 0 9 ... 3 9 2]<jupyter_text>Save the data to csvWe create a dataframe to save our predicted values in the provided csv format for submission.<jupyter_code>final_classes=[(idx+1,val) for idx, val in enumerate(classes_evaluate)]
classes_df = pd.DataFrame.from_records(final_classes, columns=['ImageId', 'Label'])
classes_df.tail(10)
classes_df.to_csv("mnist-submission.csv", index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1077016557800395, 'fraction_numerical': 0.0261279052103904, 'mean_word_length': 5.801652892561983, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '18842', 'license': 'MIT', 'n_tokens_mistral': 2034, 'n_tokens_neox': 1891, 'n_words': 570, 'path': 'MNIST/MNIST Notebook.ipynb', 'repo_name': 'manish/Kaggling', 'size': '6583'}