<jupyter_start><jupyter_text>**OCR Model Training** **Installing and Importing Required Libraries**---1. OpenCV - Image Processing Library2. Tensorflow 2.0 - Machine learning Library<jupyter_code>from IPython.display import clear_output
import cv2 
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import os

clear_output()<jupyter_output><empty_output><jupyter_text>**Getting The DATASET from the repository**---This will clone git repository. The hierarchy of this repository as shown below:          OCR -+-> data -----+--> a               |             +--> b               +-> other     :                   files     :                             +--> noise<jupyter_code>!git clone https://github.com/abhinandanarya06/OCR.git<jupyter_output>Cloning into 'OCR'...
remote: Enumerating objects: 143, done.[K
remote: Counting objects: 100% (143/143), done.[K
remote: Compressing objects: 100% (125/125), done.[K
remote: Total 19489 (delta 40), reused 65 (delta 17), pack-reused 19346[K
Receiving objects: 100% (19489/19489), 89.86 MiB | 34.21 MiB/s, done.
Resolving deltas: 100% (13288/13288), done.
Checking out files: 100% (48544/48544), done.<jupyter_text>**Function to show image**---"***imshow(image)***" function show image given as parameter<jupyter_code>def imshow(image):
  plt.imshow(image, cmap = 'gray', interpolation = 'bicubic')
  plt.xticks([]), plt.yticks([])  # to hide tick values on X and Y axis
  plt.show()<jupyter_output><empty_output><jupyter_text>**Mapping Letters classes to numbers**Keywords contains letters with appropriate index number mapping<jupyter_code>keywords = [chr(c) for c in range(ord('a'), ord('z')+1)]
cap = ['A', 'B', 'D', 'E', 'F', 'G', 'H', 'I', 'L', 'M', 'N', 'Q', 'R', 'T']
keywords = keywords + cap
keywords = keywords + ['noise']
print('No of text classification classes taken : ', len(keywords))<jupyter_output>No of text classification classes taken :  41<jupyter_text>**Preparing Data for Training**---Using Tensorflow input data pipeline API (tf.data)<jupyter_code>data_images = list()
labels = list()
i = 0
for c in keywords:
    path = 'OCR/data/{}/'.format(c) # IF YOU WANT TO GET DATA FROM OTHER PATH,
    files = os.listdir(path)        # THEN PLEASE MODIFY "path" VARIABLE ACCORDINGLY
    for name in files:
        img = cv2.imread(path+name, 0)
        try:
            img = cv2.resize(img, (30, 30), interpolation = cv2.INTER_AREA)
        except:
            continue
        img = img / np.max(img.reshape(900)) # Pixel feature normalization
        data_images.append([img])
        labels.append(i)
    i += 1<jupyter_output><empty_output><jupyter_text>**Converting data to numpy array for reshaping**<jupyter_code>data_images = np.array(data_images)
labels = np.array(labels)

DATASET_SIZE = labels.shape[0]
IMAGE_DATA_SHAPE = (DATASET_SIZE, 30, 30, 1)

data_images = data_images.reshape(IMAGE_DATA_SHAPE)
print('Image Data Shape : ', data_images.shape)
print('Labels Data Shape : ', labels.shape)<jupyter_output>Image Data Shape :  (48528, 30, 30, 1)
Labels Data Shape :  (48528,)<jupyter_text>**Converting to tf.data.Dataset type**---<jupyter_code>BATCH_SIZE = 1000

dataset = tf.data.Dataset.from_tensor_slices((data_images, labels))
dataset = dataset.shuffle(DATASET_SIZE)

VAL_SIZE = int(DATASET_SIZE * 0.2)

val_data = dataset.take(VAL_SIZE).batch(VAL_SIZE)
train_data = dataset.batch(BATCH_SIZE)<jupyter_output><empty_output><jupyter_text>**Looking into dataset**---<jupyter_code>for img, label in train_data.take(10):
  imshow(img[0].numpy().reshape((30, 30))*255)
  print(' '*12, keywords[label[0]], end='\n\n')<jupyter_output><empty_output><jupyter_text>**Text Detector Model Architecture**---Layer Sequence (Sequencial Model) : 1.   Conv2D Layer2.   Flatten Operation3.   Dense Layer with 200 units4.   10% neuron Dropout Operation5.   Dense Layer with keyword length units<jupyter_code>model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(8, (5, 5), activation='relu', input_shape = (30, 30, 1)),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(200, activation = 'relu'),
    tf.keras.layers.Dropout(0.1),
    tf.keras.layers.Dense(len(keywords))
])
model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 26, 26, 8)         208       
_________________________________________________________________
flatten (Flatten)            (None, 5408)              0         
_________________________________________________________________
dense (Dense)                (None, 200)               1081800   
_________________________________________________________________
dropout (Dropout)            (None, 200)               0         
_________________________________________________________________
dense_1 (Dense)              (None, 41)                8241      
=================================================================
Total params: 1,090,249
Trainable params: 1,090,249
Non-trainable params: 0
______________________________________________[...]<jupyter_text>**Training starts with callback of early stopping with patience 3**---<jupyter_code>callback = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=2)

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])
history = model.fit(train_data, epochs=10, callbacks = [callback], validation_data=val_data)
clear_output()<jupyter_output><empty_output><jupyter_text>**Training Analysis**---<jupyter_code>plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

# Plot training & validation loss values
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()<jupyter_output><empty_output><jupyter_text>------  **OCR Model Testing**  **Saving the trained model**---'model' model is saved in current directory with name 'model.hd5'<jupyter_code>model.save('model.h5')
clear_output()<jupyter_output><empty_output><jupyter_text>**Load Model**---from github repo<jupyter_code>model = tf.keras.models.load_model('OCR/model.h5') # change it for your preference<jupyter_output><empty_output><jupyter_text>**OCR Deployment Demo (Testing Model)**--- **Some Conventions**<jupyter_code>X = 0   # X coordinate of the character in image space
Y = 1   # Y coordinate of the character in image space
POS = 1    # shows and return the (x,y) coordinate of image space
SHAPE = 2  # shows and return the (width, height) of character contour in image space
W = 0 # width of the character
H = 1 # height of the character<jupyter_output><empty_output><jupyter_text>"***check_in(c, region)***" checks if character is present in the given region "*region*" of the image space<jupyter_code>def check_in(c, region):
    x, y, w, h = region
    center_x = c[POS][X]+c[SHAPE][W]/2
    center_y = c[POS][Y]
    if (center_x > x-1 and center_x < x+w+1) and (center_y > y-1 and center_y < y+h+1):
        return True
    return False<jupyter_output><empty_output><jupyter_text>"***get_region(c, regions)***" finds the region in the given list of regions "*regions*" in image space and return it<jupyter_code>def get_region(c, regions):
    for region in regions:
        if check_in(c, region):
            return region
    return False<jupyter_output><empty_output><jupyter_text>"***sort_chars(line)***" Function to sort characters in given line<jupyter_code>def sort_chars(line):
    return sorted(line, key=lambda c: c[POS][X])<jupyter_output><empty_output><jupyter_text>"***sort_lines_by_yval(lines)***" function sorts list of lines according to its y coordinate in image space<jupyter_code>def sort_lines_by_yval(lines):
    return sorted(lines, key=lambda line: line[0][POS][Y])<jupyter_output><empty_output><jupyter_text>"***group_chars_by_line(characters)***" Function to group characters in given image by lines<jupyter_code>def group_chars_by_line(characters):
  lines = list()
  while len(characters) > 0:
      m = characters[0]
      my = m[1][1]
      my_plus_h = m[1][1]+m[2][1]
      lines.append([m])
      for c in characters[1:]:
          if my <= c[POS][Y]+c[SHAPE][H]/2 and c[POS][Y]+c[SHAPE][H]/2 <= my_plus_h:
              if my > c[POS][Y]:
                  my = c[POS][Y]
              if my_plus_h < c[POS][Y]+c[SHAPE][H]:
                  my_plus_h = c[POS][Y]+c[SHAPE][H]
              lines[-1].append(c)
              characters.remove(c)
      lines[-1]= sort_chars(lines[-1])
      characters.remove(m)
  return lines<jupyter_output><empty_output><jupyter_text>"***apply_ocr(img, text_detector_model)***" is the function that does actual ocr operation on image parameter to get text<jupyter_code>def apply_ocr(img, text_detector):
    avg_text_height = 0 # will be assigned with Average Height of text character
    character_list = list() # will contain necessary text character to be extracted
    img = cv2.medianBlur(img,5)
    img = cv2.adaptiveThreshold(img,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C,\
            cv2.THRESH_BINARY,41,10) # will remove maximum noise and make image suitable for contour dt
    contours, hierarchy = cv2.findContours(img,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
    black_map = img.copy()
    black_map[:, :] = 0
    i = 0
    for contour in contours:
        try:
            x,y,w,h = cv2.boundingRect(contour)
            try:
                cnt = img[y-2:y+h+2, x-2:x+w+2]
            except:
                cnt = img[y:y+h, x:x+w]
            cnt = cv2.resize(cnt, (30, 30), interpolation = cv2.INTER_AREA)
            cnt = cnt.reshape((1, 30, 30, 1))
            cnt = cnt / np.max(cnt.reshape(900))
            class_pred = np.argmax(text_detector.predict(cnt))
            if class_pred < 40:
                avg_text_height += h
                cv2.rectangle(black_map, (x-w//7,y), (x+w+w//7, y+h), 255, -1)
                character_list.append([keywords[class_pred], (x,y), (w,h)])
                i += 1
        except:
              continue
    avg_text_height /= i # Average Height of text character
    text = group_chars_by_line(character_list)
    text = sort_lines_by_yval(text)
    contours, hierarchy = cv2.findContours(black_map, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
    regions = list() # will contain regions of words for grouping chars by word
    for contour in contours:
        region = cv2.boundingRect(contour)
        if region[SHAPE + H] <= avg_text_height*7:
          regions.append(region)
    del contours

    TEXT = ''
    for l in text:
        char = l[0]
        region = get_region(char, regions)
        if not region:
            continue
        for char in l:
            if not check_in(char, region):
                TEXT = TEXT + ' '
                r = get_region(char, regions)
                if not r:
                    continue
                region = r
            TEXT = TEXT + char[0]
        TEXT = TEXT + '\n'
    return TEXT<jupyter_output><empty_output><jupyter_text>**Main Run Start Point**---* Before running this start point, Please check if  *test_image_path* variable have path to your own test images or not.* If not, please modify the path variable<jupyter_code>test_images_path = 'OCR/sample_test_image/' # IF YOU PLACE TEST IMAGE INSIDE OCR DIRECTORY,
                                            # THEN PLEASE MODIFY "test_images_path" variable


imgs = [f for f in os.listdir(test_images_path) if f.endswith('.jpg')]

for img in imgs:
    print('*'*30, 'Text on {}'.format(img), '*'*30)
    img = cv2.imread(test_images_path + img, 0)
    TEXT = apply_ocr(img, model)
    imshow(img)
    print(TEXT)
    print('-'*80, '\n\n')<jupyter_output>****************************** Text on 2-cropped.jpg ******************************
{'fraction_non_alphanumeric': 0.14380858411445485, 'fraction_numerical': 0.02631146193060352, 'mean_word_length': 3.6106899166034876, 'pattern_counts': {'":': 0, '<': 87, '<?xml version=': 0, '>': 89, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '21', 'dir': 'jupyter-structured-clean-dedup', 'id': '416516', 'license': 'MIT', 'n_tokens_mistral': 4181, 'n_tokens_neox': 3881, 'n_words': 960, 'path': 'Research_Notebook_on_OCR.ipynb', 'repo_name': 'abhinandanarya06/OCR', 'size': '12162'}