We present an end-to-end trainable approach for optical character recognition (OCR) on printed documents. It is based on predicting a two-dimensional character grid ('chargrid') representation of a document image as a semantic segmentation task.
 To identify individual character instances from the chargrid, we regard characters as objects and use object detection techniques from computer vision.
 We demonstrate experimentally that our method outperforms previous state-of-the-art approaches in accuracy while being easily parallelizable on GPU (thereby being significantly faster), as well as easier to train. Optical Character Recognition (OCR) on documents is an age-old problem for which numerous open-source (e.g. [14] ) as well as proprietary solutions exist. Especially in the sub-domain of printed documents, it is often regarded as being solved. However, current state-of-the-art document-level OCR solutions (as far as the published research goes) consist of a complicated pipeline of steps, each one either a hand-optimized heuristic or requiring intermediate data and annotations to train. Deep neural networks have been proven very successful in object detection tasks [8] . In this work, we build on top of these developments and treat OCR as a semantic segmentation and object detection task for detecting and recognizing character instances on a page. 2 We introduce a new end-toend trainable OCR pipeline for (but not limited to) printed documents that is based on deep fully convolutional neural networks. Our main contribution is to frame the OCR problem as an ultra-dense instance-segmentation task [5] for characters over the full input document image. We do not rely on any pre-processing stages like binarization, deskewing, layout analysis. Instead, our model learns directly from the raw document pixel data. At the core of our method, we predict a chargrid representation [6] of the input document -a 1-hot encoded grid of characters. Thus, we call our method Chargrid-OCR. Additionally, we introduce two novel post-processing steps, both of which are crucial to performing fast and accurate dense OCR. We show that our method can outperform line-based pipelines like e.g. Tesseract 4 [13] that rely on a combination of deep convolutional and recurrent networks with CTC loss [14, 1] while being significantly simpler to train. * Equal contribution 2 A related task of recognizing text in natural images, referred to as Scene Text Recognition (STR), has been faster in adopting techniques from object detection in computer vision [3] . However, compared to STR, document OCR deals with much denser text and very high accuracy requirements [2] . 2 Chargrid-OCR: OCR as an ultra-dense object detection task Chargrid-OCR method is a lexicon-free (only character-based), end-to-end trainable approach for OCR. Given a document image, chargrid-OCR predicts character segmentation mask together with object bounding boxes for characters in one single step (see Fig 1) . Both, semantic segmentation and object detection are common tasks in computer vision, e.g. [11, 8, 7] . The character segmentation mask classifies each pixel into a character class and the character bounding box detects a bounding box around each character. Both, our semantic segmentation and box detection (sub-)networks are fully convolutional and consist of only a single stage (like [8] and unlike [9] ). Being single-stage is especially important as there may be thousands of characters (i.e. objects) on a single page which yields an ultra-dense object detection task. We presented a new end-to-end trainable optical character recognition pipeline that is based on state-of-the-art computer vision approaches using object detection and semantic segmentation. Our pipeline is significantly simpler compared to other sequential and line-based approaches, especially those used for document-level optical character recognition such as Tesseract 4. We empirically show that our model outperforms Tesseract 4 on a number of diverse evaluation datasets by a large margin both in terms of accuracy and run-time.
End-to-end trainable Optical Character Recognition on printed documents; we achieve state-of-the-art results, beating Tesseract4 on benchmark datasets both in terms of accuracy and runtime, using a purely computer vision based approach.
{"src": "scitldr-AIC_train", "paper_id": "SkxhzT5qIB"}