<jupyter_start><jupyter_text>Recognizing Persian Digits  Introduction to problemGiven images of Persian Digits with their labels, predict what is the digit inside each image: ![digit_image](figs/8-Digit-Recognition.png)  Contents* Libraries & Constants* Machine Learning Categories* Algorithm Selection* Loading and Preprocessing * Model Impelemtation* Training* Model Evaluation* Prediction  Libraries & Constants<jupyter_code>from __future__ import print_function

import matplotlib.pyplot as plt
import numpy as np
import os
import sys
import tarfile
import random
import utils
from PIL import Image
import tensorflow as tf
from IPython.display import display
from scipy import ndimage
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from six.moves.urllib.request import urlretrieve
from six.moves import cPickle as pickle
%matplotlib inline
root = './data/Digits/isolated digits/'
prediction_root = './data/Digits/segmented digits/'
binary_save_path = './binary_data/isolated digits/'
model_save_path = './models/'
num_labels= 10
image_size = 28
batch_size = 64
epochs = 40
hidden_layer_neurons = 15<jupyter_output><empty_output><jupyter_text>Machine Learning Categories* Supervised Learning* Unsupervised Learning* Semi-supervised Learning* Reinforcement Learning  Algorithm Selection    Linear Regression    Logistic Regression    Decision Tree    Neural Networks    Naive Bayes    SVM    kNN    Etc... We choose neural networks because of:* Number of features: 28*28=784* Experiments showed that Neural Networks are good at classifying images![neural_network](figs/Neural-network-with-hidden-layer-for-MNIST-data.png)  Loading and Preprocessing DataFirst we will load and normalize data, then we resize all of the images in (28,28) size and save it to binary .npy file:<jupyter_code>data_paths = utils.maybe_save(root,binary_save_path,image_size)<jupyter_output>./binary_data/isolated digits/0.npy already present - Loading Data.
./binary_data/isolated digits/1.npy already present - Loading Data.
./binary_data/isolated digits/2.npy already present - Loading Data.
./binary_data/isolated digits/3.npy already present - Loading Data.
./binary_data/isolated digits/4.npy already present - Loading Data.
./binary_data/isolated digits/5.npy already present - Loading Data.
./binary_data/isolated digits/6.npy already present - Loading Data.
./binary_data/isolated digits/7.npy already present - Loading Data.
./binary_data/isolated digits/8.npy already present - Loading Data.
./binary_data/isolated digits/9.npy already present - Loading Data.<jupyter_text>Checking if data is Balanced across diffrent classes :<jupyter_code>def balancingCheck(data_folder) :
    dictionary = {}
    tmpList = None
    for i,ds in enumerate(data_folder) :
        dataset = np.load(ds)
        dictionary[i] = len(dataset)
    plt.bar(range(len(dictionary)), list(dictionary.values()), align='center')
    plt.xticks(range(len(dictionary)), list(dictionary.keys()))
    plt.show()
balancingCheck(data_paths)<jupyter_output><empty_output><jupyter_text>As you see data is almost balanced over diffrent classes. Merging and Creating Label for datasets :<jupyter_code>def merge_datasets(data_paths) :    
    data = np.concatenate([np.load(path) for path in data_paths],axis=0)
    label = np.concatenate([[path[-5]]*len(np.load(path)) for i,path in enumerate(data_paths)])
    return data,label
data,labels = merge_datasets(data_paths)<jupyter_output><empty_output><jupyter_text>We should shuffle data:<jupyter_code>def randomize(dataset, labels):
    permutation = np.random.permutation(labels.shape[0])
    shuffled_dataset = dataset[permutation,:,:]
    shuffled_labels = labels[permutation]
    return shuffled_dataset, shuffled_labels
data,labels = randomize(data,labels)<jupyter_output><empty_output><jupyter_text>We want to make sure that data still looks good:<jupyter_code>def visualize(dataset,labels,size = 10) :
    samples = np.random.randint(len(dataset),size=size)
    for i,sample in enumerate(samples) :
        plt.subplots_adjust(top=1.5)
        plt.subplot(np.ceil(np.sqrt(size)),np.floor(np.sqrt(size)),1+i)
        plt.axis('off')
        plt.title(labels[sample])
        plt.imshow(dataset[sample])
visualize(data,labels,10)
def reformat(data,labels) :
    data = np.array(data).reshape((-1,image_size*image_size)).astype(np.float32)
    labels = (np.arange(num_labels) == labels[:,None].astype(np.float32))
    return data,labels
data,labels = reformat(data,labels)
print('shape of data : {}'.format(data.shape))
print('shape of labels : {}'.format(labels.shape))<jupyter_output>shape of data : (9653, 784)
shape of labels : (9653, 10)<jupyter_text>![train_validate_test](figs/train-validate-test.png)<jupyter_code>def train_valid_test_spliter(data,labels) :
    X_train, X_test, y_train, y_test = train_test_split(data, labels, test_size=0.2, random_state=1)
    X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.2, random_state=1)
    return X_train,y_train,X_val,y_val,X_test,y_test
X_train,y_train,X_val,y_val,X_test,y_test = train_valid_test_spliter(data,labels)
print('train data shape: {}'.format(X_train.shape))
print('train labels shape: {}'.format(X_train.shape))
print('valid data shape : {}'.format(X_val.shape))
print('valid labels shape: {}'.format(y_val.shape))
print('test data shape : {}'.format(X_test.shape))
print('test labels shape: {}'.format(y_test.shape))<jupyter_output>train data shape: (6177, 784)
train labels shape: (6177, 784)
valid data shape : (1545, 784)
valid labels shape: (1545, 10)
test data shape : (1931, 784)
test labels shape: (1931, 10)<jupyter_text>Model Implementation<jupyter_code>model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(hidden_layer_neurons, activation='relu', input_shape=(image_size*image_size,)))
model.add(tf.keras.layers.Dense(num_labels, activation='softmax'))
model.compile(optimizer=tf.keras.optimizers.Adam(lr=1e-4), loss='categorical_crossentropy', metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>Training<jupyter_code>history = model.fit(X_train,y_train,batch_size=batch_size,epochs=epochs,validation_data=(X_val,y_val),callbacks=callbacks)<jupyter_output>Train on 6177 samples, validate on 1545 samples
Epoch 1/40
5120/6177 [=======================>......] - ETA: 0s - loss: 0.3377 - acc: 0.9109{'val_loss': 0.3498847692819089, 'val_acc': 0.9087378640776699, 'loss': 0.33815116498855124, 'acc': 0.9114456855886014}
0
6177/6177 [==============================] - 0s 56us/step - loss: 0.3382 - acc: 0.9114 - val_loss: 0.3499 - val_acc: 0.9087
Epoch 2/40
5568/6177 [==========================>...] - ETA: 0s - loss: 0.3333 - acc: 0.9106{'val_loss': 0.348442464526803, 'val_acc': 0.9061488673139159, 'loss': 0.3346867349293696, 'acc': 0.9106362311705352}
1
6177/6177 [==============================] - 0s 62us/step - loss: 0.3347 - acc: 0.9106 - val_loss: 0.3484 - val_acc: 0.9061
Epoch 3/40
5696/6177 [==========================>...] - ETA: 0s - loss: 0.3325 - acc: 0.9124{'val_loss': 0.34502619982344435, 'val_acc': 0.9093851132686084, 'loss': 0.3313438591928916, 'acc': 0.9125789219321451}
2
6177/6177 [==============================] - 0s 54us/step - loss[...]<jupyter_text>Visualizing the training and validation accuracy:<jupyter_code>plt.plot(history.epoch, history.history['val_acc'], c='r')
plt.plot(history.epoch, history.history['acc'], c='g')
plt.title('Training & Validation Accuracy')
plt.xlabel('Epoch Number')
plt.ylabel('Accuracy')
plt.legend(('Validation Accuracy', 'Training Accuracy'),
           loc='lower right')<jupyter_output><empty_output><jupyter_text>Model Evaluation<jupyter_code>scores = model.evaluate(X_test,y_test)
print('Test Accuracy: %{}'.format(scores[1]*100))
print("Test Error: %{}".format(100 - scores[1]*100))<jupyter_output>1931/1931 [==============================] - ETA:  - 1s 276us/step
Test Accuracy: %88.60693940345887
Test Error: %11.393060596541133<jupyter_text>Prediction<jupyter_code>def predict_image(prediction_root,image_size,model) :
    for i,path in enumerate(os.listdir(prediction_root)):
        plt.subplots_adjust(top=1.5)
        photoes_path = os.path.join(prediction_root,path)
        photoes = os.listdir(photoes_path)
        photo_path = np.random.choice(photoes)
        processed_image = (np.array(Image.open(os.path.join(photoes_path, photo_path)).resize((image_size,image_size))).astype(float) - 
                            255.0 / 2) / 255.0
        photo = np.array(Image.open(os.path.join(photoes_path,photo_path)).resize((image_size,image_size))).astype(np.float)
        plt.subplot(np.ceil(np.sqrt(10)),np.floor(np.sqrt(10)),1+i)
        plt.axis('off')
        plt.title(np.argmax(model.predict(processed_image.reshape(1,image_size*image_size))))
        plt.imshow(photo)
predict_image(prediction_root,image_size,model)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1481730553687514, 'fraction_numerical': 0.06007621609504595, 'mean_word_length': 5.703981968444778, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '182508', 'license': 'MIT', 'n_tokens_mistral': 3456, 'n_tokens_neox': 3027, 'n_words': 612, 'path': 'First Meetup/code/notebook.ipynb', 'repo_name': 'school-of-ai-rasht-chapter/Meetup-Materials', 'size': '8922'}