Imagery texts are usually organized as a hierarchy of several visual elements, i.e. characters, words, text lines and text blocks. Among these elements, character is the most basic one for various languages such as Western, Chinese, Japanese, mathematical expression and etc. It is natural and convenient to construct a common text detection engine based on character detectors. However, training character detectors requires a vast of location annotated characters, which are expensive to obtain. Actually, the existing real text datasets are mostly annotated in word or line level. To remedy this dilemma, we propose a weakly supervised framework that can utilize word annotations, either in tight quadrangles or the more loose bounding boxes, for character detector training. When applied in scene text detection, we are thus able to train a robust character detector by exploiting word annotations in the rich large-scale real scene text datasets, e.g. ICDAR15 and COCO-text. The character detector acts as a key role in the pipeline of our text detection engine. It achieves the state-of-the-art performance on several challenging scene text detection benchmarks. We also demonstrate the flexibility of our pipeline by various scenarios, including deformed text detection and math expression recognition.
Recently, some component based methods @cite_50 @cite_45 @cite_23 @cite_0 @cite_51 attempt to learn text components by CNN feature learning. The components are either representative pixels @cite_50 @cite_45 @cite_23 or segment boxes @cite_0 @cite_51 . These methods can learn from word annotations. In addition, text component is also a basic visual element, which may also benefit a common text detection engine. Nevertheless, our method takes advantages over these methods in the following aspects: first, characters provide stronger cues, e.g., character scales and center locations, for the subsequent text structure analysis module; second, character is a semantic element, while component not. Thus our method is applicable to problems where direct character recognition is needed, e.g. match expression; third, our method can utilize loose word annotations for training, e.g. bounding box annotations in the COCO-Text dataset @cite_27 . This is because our method can refine character center labels during training. For the above component based methods, their noisy labels are fixed which may harm training.
@cite_0 @cite_27 @cite_45 @cite_23 @cite_50 @cite_51
2472159136 2749425057 2751748110 2783835666
We propose a system that finds text in natural scenes using a variety of cues. Our novel data-driven method incorporates coarse-to-fine detection of character pixels using convolutional features (Text-Conv), followed by extracting connected components (CCs) from characters using edge and color features, and finally performing a graph-based segmentation of CCs into words (Word-Graph). For Text-Conv, the initial detection is based on convolutional feature maps similar to those used in Convolutional Neural Networks (CNNs), but learned using Convolutional k-means. Convolution masks defined by local and neighboring patch features are used to improve detection accuracy. The Word-Graph algorithm uses contextual information to both improve word segmentation and prune false character word detections. Different definitions for foreground (text) regions are used to train the detection stages, some based on bounding box intersection, and others on bounding box and pixel intersection. Our system obtains pixel, character, and word detection f-measures of 93.14 , 90.26 , and 86.77 respectively for the ICDAR 2015 Robust Reading Focused Scene Text dataset, out-performing state-of-the-art systems. This approach may work for other detection targets with homogenous color in natural scenes. Imagery texts are usually organized as a hierarchy of several visual elements, i.e. characters, words, text lines and text blocks. Among these elements, character is the most basic one for various languages such as Western, Chinese, Japanese, mathematical expression and etc. It is natural and convenient to construct a common text detection engine based on character detectors. However, training character detectors requires a vast of location annotated characters, which are expensive to obtain. Actually, the existing real text datasets are mostly annotated in word or line level. To remedy this dilemma, we propose a weakly supervised framework that can utilize word annotations, either in tight quadrangles or the more loose bounding boxes, for character detector training. When applied in scene text detection, we are thus able to train a robust character detector by exploiting word annotations in the rich large-scale real scene text datasets, e.g. ICDAR15 and COCO-text. The character detector acts as a key role in the pipeline of our text detection engine. It achieves the state-of-the-art performance on several challenging scene text detection benchmarks. We also demonstrate the flexibility of our pipeline by various scenarios, including deformed text detection and math expression recognition. Scene text recognition has attracted great interests from the computer vision and pattern recognition community in recent years. State-of-the-art methods use concolutional neural networks (CNNs), recurrent neural networks with long short-term memory (RNN-LSTM) or the combination of them. In this paper, we investigate the intrinsic characteristics of text recognition, and inspired by human cognition mechanisms in reading texts, we propose a scene text recognition method with character models on convolutional feature map. The method simultaneously detects and recognizes characters by sliding the text line image with character models, which are learned end-to-end on text line images labeled with text transcripts. The character classifier outputs on the sliding windows are normalized and decoded with Connectionist Temporal Classification (CTC) based algorithm. Compared to previous methods, our method has a number of appealing properties: (1) It avoids the difficulty of character segmentation which hinders the performance of segmentation-based recognition methods; (2) The model can be trained simply and efficiently because it avoids gradient vanishing exploding in training RNN-LSTM based models; (3) It bases on character models trained free of lexicon, and can recognize unknown words. (4) The recognition process is highly parallel and enables fast recognition. Our experiments on several challenging English and Chinese benchmarks, including the IIIT-5K, SVT, ICDAR03 13 and TRW15 datasets, demonstrate that the proposed method yields superior or comparable performance to state-of-the-art methods while the model size is relatively small. Text recognition from natural scene images is very tough task now these days compare than videos. Application of image processing called pattern recognition make easy to recognize text from multimedia documents. A pattern can be fingerprint image, handwritten word sample, human face images, speech signal and DNA sequence etc or we can say that all pattern are in machine editable form. Text can be recognized with and without segmentation of character. Segmentation can be line, word or character level and without segmentation character is recognized from whole text image. Character recognition is a field of research and various research has been done in the area of pattern recognition. There we use a new technique called diagonal based feature extraction in last layer of convolutional neural network and make feature extraction easy with the help of genetic algorithm. After extraction of feature we provide training to extreme learning machine. Along this feature extraction technique we use feed forward network as a classifier and convolution neural network for feature extractor. It is a deep learning based technique of neural network which use for classification or recognition of text. This is basically used for providing training and in testing phase. CRConvNet has more layers working of all layer shown in flowchart. One dataset which contain 360 training set data that are all in capital(A-Z) and small(a-z) alphabet, digit (0–9) and some special character are also used. Another dataset contains samples of video and images (ICDAR 2003) for testing. Extensive studies shows that the recognition system which using diagonal based feature learning provide high recognition accuracy while requiring less time for training.