<jupyter_start><jupyter_text>Predicting numbers using deepNets in MNIST dataset Installing deepNetsFirst we need to install deepNets package<jupyter_code>!pip install deepNets<jupyter_output>Requirement already satisfied: deepNets in /usr/local/lib/python3.7/dist-packages (0.1.7)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from deepNets) (1.19.5)<jupyter_text>Import necessary packagesSince we need the MNIST dataset make sure you have the folder, it's present by default in Google Colab or else you can simply download and move it your juypter notebook<jupyter_code>from deepNets import Net                              #for constructing networks
import pandas as pd                                   #for reading
import numpy as np                                    #for handling arrays
from sklearn.model_selection import train_test_split  #for train-test split

train = pd.read_csv("/content/sample_data/mnist_train_small.csv")
test = pd.read_csv("/content/sample_data/mnist_test.csv")<jupyter_output><empty_output><jupyter_text>*Splitting target and features*<jupyter_code>#training data
y = train.iloc[:,0]
X = train.drop(train.columns[0],axis=1)
#test data
y_test = test.iloc[:,0]
X_test = test.drop(test.columns[0],axis=1)
#View training labels
y.value_counts()<jupyter_output><empty_output><jupyter_text>*Some details about our feature variables*<jupyter_code>print(X.isnull().any().describe(),"\n",X_test.isnull().any().describe(),sep="")<jupyter_output>count       784
unique        1
top       False
freq        784
dtype: object
count       784
unique        1
top       False
freq        784
dtype: object<jupyter_text>Data-preprocessing*   Normalizing input to be in the range of 0-1 since we're dealing with classification problem. *   Dividing by the max value of each channel (red, green, blue) , since it's a image it will be between 0-255*   Note that we're using the shape CxHxW where C - no of channels , H - height of the image and W - width of the image.*Since the MNIST dataset is represented in greyscale the number of channels is 1*<jupyter_code>X = X/255.0
X_test = X_test/255.0
X = X.values.reshape(-1,1,28,28)
X_test = X_test.values.reshape(-1,1,28,28)
y = y.to_numpy()
y_test = y_test.to_numpy()<jupyter_output><empty_output><jupyter_text>**Splitting into training and validation data****Visualization of the 16th element in the dataset**<jupyter_code>import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import seaborn as sns
%matplotlib inline

X_train,X_val,y_train,y_val= train_test_split(X,y,test_size = 0.2,random_state = 2)  #Splitting by 80% train & 20% validation
plt.imshow(X_train[16][0,:,:])<jupyter_output><empty_output><jupyter_text>Constucting our neural network structure using deepNetsThe layers are simply a list/array of dictionaries where each dictionary may be different.You don't need to worry about the gradients deepNets takes care of that***Here the order of how you specify the dictionary inside the list is important***<jupyter_code>make_layers = []
make_layers.append({"layer_type":"input","inp":X_train})
make_layers.append({"layer_type":"conv","filters":32,"filter_size":5,"padding":2})
make_layers.append({"layer_type":"batchnorm"})
make_layers.append({"layer_type":"pool","filter_size":2,"stride":2})
make_layers.append({"layer_type":"conv","filters":16,"filter_size":3,"padding":2})
make_layers.append({"layer_type":"batchnorm"})
make_layers.append({"layer_type":"pool","filter_size":2,"stride":2})
make_layers.append({"layer_type":"loss","num_classes":10})
net = Net.Net()             #The Net class helps in constructing the network and initializing your weights
#net.reg = 0.25             #incase you need regularization
net.makeLayers(make_layers) #Net() takes care of the computational graph and makeLayers() makes the initialization part<jupyter_output><empty_output><jupyter_text>Training our neural netHere you can see that we're using a Trainer helper function of deepNets which basically does all the training and predicting accuracy part. All you need to do is simply feed in the training and validation data and our neural net model (in our case is called as "net") along with batch_size, learning rate and type of update<jupyter_code>from deepNets import Trainer as trn

trainer = trn.Trainer(net,X_train,y_train,(X_val,y_val),update_rule="sgd_momentum" \
                      ,lr_decay=0.95,optim_config={'learning_rate':0.001},batch_size=100,verbose=False)
trainer.train()<jupyter_output><empty_output><jupyter_text>Let's see our best validation score<jupyter_code>print("Best validation accuracy: ","{:0.2%}".format(max(trainer.val_acc_history)))
plt.plot(trainer.loss_history)
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epochs')
plt.legend(['Loss'])
plt.show()<jupyter_output>Best validation accuracy:  98.10%<jupyter_text>Testing our model on Test set<jupyter_code>import random

predict_scores = net.loss(X_test)                 #this uses the best weigths by default
y_pred = list(np.argmax(predict_scores,axis=1))   #Finding the column with highest score taking it's index (since our target variables are indexes)
random_numbers = random.sample(range(0,100),1)    #Feel free to change the second parameter and test it
for number in random_numbers:
  plt.figure()
  plt.imshow(X_test[number][0,:,:])
  print("Predicted: ",y_pred[number])
  print("Correct: ")<jupyter_output>Predicted:  4
Correct:
{'fraction_non_alphanumeric': 0.10982552800734618, 'fraction_numerical': 0.020385674931129475, 'mean_word_length': 5.36214953271028, 'pattern_counts': {'":': 20, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '117865', 'license': 'MIT', 'n_tokens_mistral': 1812, 'n_tokens_neox': 1723, 'n_words': 483, 'path': 'MNISTdeepNets.ipynb', 'repo_name': 'DeepakVelmurugan/deepNets', 'size': '5445'}