We propose a new framework for the recognition of online handwritten graphics. Three main features of the framework are its ability to treat symbol and structural level information in an integrated way, its flexibility with respect to different families of graphics, and means to control the tradeoff between recognition effectiveness and computational cost. We model a graphic as a labeled graph generated from a graph grammar. Non-terminal vertices represent subcomponents, terminal vertices represent symbols, and edges represent relations between subcomponents or symbols. We then model the recognition problem as a graph parsing problem: given an input stroke set, we search for a parse tree that represents the best interpretation of the input. Our graph parsing algorithm generates multiple interpretations (consistent with the grammar) and then we extract an optimal interpretation according to a cost function that takes into consideration the likelihood scores of symbols and structures. The parsing algorithm consists in recursively partitioning the stroke set according to structures defined in the grammar and it does not impose constraints present in some previous works (e.g. stroke ordering). By avoiding such constraints and thanks to the powerful representativeness of graphs, our approach can be adapted to the recognition of different graphic notations. We show applications to the recognition of mathematical expressions and flowcharts. Experimentation shows that our method obtains state-of-the-art accuracy in both applications.
In this section, we review some characteristics of the recognition process in previous works, with emphasis on methods for mathematical expression @cite_0 @cite_17 @cite_30 and flowchart recognition @cite_38 @cite_12 @cite_6 .
@cite_30 @cite_38 @cite_6 @cite_0 @cite_12 @cite_17
2031071334 2013257774 2031948510 2037121285
Document recognition and retrieval technologies complement one another, providing improved access to increasingly large document collections. While recognition and retrieval of textual information is fairly mature, with wide-spread availability of optical character recognition and text-based search engines, recognition and retrieval of graphics such as images, figures, tables, diagrams, and mathematical expressions are in comparatively early stages of research. This paper surveys the state of the art in recognition and retrieval of mathematical expressions, organized around four key problems in math retrieval (query construction, normalization, indexing, and relevance feedback), and four key problems in math recognition (detecting expressions, detecting and classifying symbols, analyzing symbol layout, and constructing a representation of meaning). Of special interest is the machine learning problem of jointly optimizing the component algorithms in a math recognition system, and developing effective indexing, retrieval and relevance feedback algorithms for math retrieval. Another important open problem is developing user interfaces that seamlessly integrate recognition and retrieval. Activity in these important research areas is increasing, in part because math notation provides an excellent domain for studying problems common to many document and graphics recognition and retrieval applications, and also because mature applications will likely provide substantial benefits for education, research, and mathematical literacy. In order to segment and recognize on-line handwritten flowchart symbols precisely, we propose a method that segments the graphic symbols based on the loop structure and recognize the segmented symbols by using SVMs. In our experiments, low error rate of 3.37 for symbol segmentation and high recognition rate of 97.6 were obtained. We also propose a beautification and editing method for recognized symbols, and implement them to construct a prototype system. We compare an input time for drawing flowcharts between our system and a traditional application using icon-based interface. As a result, the input time on our system was faster than that on traditional one for flowcharts without texts. We present our recent model of a diagram recognition engine. It extends our previous work which approaches the structural recognition as an optimization problem of choosing the best subset of symbol candidates. The main improvement is the integration of our own text separator into the pipeline to deal with text blocks occurring in diagrams. Second improvement is splitting the symbol candidates detection into two stages: uniform symbols detection and arrows detection. Text recognition is left for post processing when the diagram structure is already known. Training and testing of the engine was done on a freely available benchmark database of flowcharts. We correctly segmented and recognized 93.0 of the symbols having 55.1 of the diagrams recognized without any error. Considering correct stroke labeling, we achieved the precision of 95.7 . This result is superior to the state-of-the-art method with the precision of 92.4 . Additionally, we demonstrate the generality of the proposed method by adapting the system to finite automata domain and evaluating it on own database of such diagrams. Automatic recognition of mathematical ex- pressions is one of the key vehicles in the drive towards transcribing documents in scientific and engineering dis- ciplines into electronic form. This problem typically con- sists of two major stages, namely, symbol recognition and structural analysis. In this survey paper, we will re- view most of the existing work with respect to each of the two major stages of the recognition process. In par- ticular, we try to put emphasis on the similarities and differences between systems. Moreover, some important issues in mathematical expression recognition will be ad- dressed in depth. All these together serve to provide a clear overall picture of how this research area has been developed to date.