We propose a new framework for the recognition of online handwritten graphics. Three main features of the framework are its ability to treat symbol and structural level information in an integrated way, its flexibility with respect to different families of graphics, and means to control the tradeoff between recognition effectiveness and computational cost. We model a graphic as a labeled graph generated from a graph grammar. Non-terminal vertices represent subcomponents, terminal vertices represent symbols, and edges represent relations between subcomponents or symbols. We then model the recognition problem as a graph parsing problem: given an input stroke set, we search for a parse tree that represents the best interpretation of the input. Our graph parsing algorithm generates multiple interpretations (consistent with the grammar) and then we extract an optimal interpretation according to a cost function that takes into consideration the likelihood scores of symbols and structures. The parsing algorithm consists in recursively partitioning the stroke set according to structures defined in the grammar and it does not impose constraints present in some previous works (e.g. stroke ordering). By avoiding such constraints and thanks to the powerful representativeness of graphs, our approach can be adapted to the recognition of different graphic notations. We show applications to the recognition of mathematical expressions and flowcharts. Experimentation shows that our method obtains state-of-the-art accuracy in both applications.
Early works related to the recognition of mathematical expressions were predominantly based on a sequential recognition process consisting of the symbol segmentation, symbol identification and structural analysis steps @cite_26 @cite_39 @cite_34 . However, a weakness of sequential methods is the fact that errors in early steps are propagated to subsequent steps. For instance, it might be difficult to determine if two handwritten strokes with shape )" and (", close to each other, form a single symbol x", or are the opening and the closing parentheses, respectively. To solve this type of ambiguity, it may be necessary to examine relations of the strokes with other nearby symbols or even with respect to the global structure of the whole expression. This type of observation has motivated more recent works to consider methods that integrate symbol and structural level interpretations into a single process. Most of them are based on parsing methods as described below.
@cite_26 @cite_34 @cite_39
1518661169 2037121285 2031948510 2015551990
In recent years, the recognition of handwritten mathematical expressions has recieved an increasing amount of attention in pattern recognition research. The diversity of approaches to the problem and the lack of a commercially viable system, however, indicate that there is still much research to be done in this area. In this thesis, I will describe an on-line approach for converting a handwritten mathematical expression into an equivalent expression in a typesetting command language such as TEX or MathML, as well as a feedback-oriented user interface which can make errors more tolerable to the end user since they can be quickly corrected. The three primary components of this system are a method for classifying isolated handwritten symbols, an algorithm for partitioning an expression into symbols, and an algorithm for converting a two-dimensional arrangements of symbols into a typeset expression. For symbol classification, a Gaussian classifier is used to rank order the interpretations of a set of strokes as a single symbol. To partition an expression, the values generated by the symbol classifier are used to perform a constrained search of possible partitions for the one with the minimum summed cost. Finally, the expression is parsed using a simple geometric grammar. Thesis Supervisor: Paul A. Viola Title: Associate Professor Automatic recognition of mathematical ex- pressions is one of the key vehicles in the drive towards transcribing documents in scientific and engineering dis- ciplines into electronic form. This problem typically con- sists of two major stages, namely, symbol recognition and structural analysis. In this survey paper, we will re- view most of the existing work with respect to each of the two major stages of the recognition process. In par- ticular, we try to put emphasis on the similarities and differences between systems. Moreover, some important issues in mathematical expression recognition will be ad- dressed in depth. All these together serve to provide a clear overall picture of how this research area has been developed to date. We present our recent model of a diagram recognition engine. It extends our previous work which approaches the structural recognition as an optimization problem of choosing the best subset of symbol candidates. The main improvement is the integration of our own text separator into the pipeline to deal with text blocks occurring in diagrams. Second improvement is splitting the symbol candidates detection into two stages: uniform symbols detection and arrows detection. Text recognition is left for post processing when the diagram structure is already known. Training and testing of the engine was done on a freely available benchmark database of flowcharts. We correctly segmented and recognized 93.0 of the symbols having 55.1 of the diagrams recognized without any error. Considering correct stroke labeling, we achieved the precision of 95.7 . This result is superior to the state-of-the-art method with the precision of 92.4 . Additionally, we demonstrate the generality of the proposed method by adapting the system to finite automata domain and evaluating it on own database of such diagrams. Recently, great progress has been made in sign language recognition. Most approaches are based on the Hidden Markov Model (HMM) with various features, such as motion trajectory. Recognition for sign sentences is obtained from optimal path by Viterbi algorithm, however, some wrong jumps are usually caused by transitional movements between signs. To address the problem, in this paper, we propose an approach consisting of two stages: offline training and online recognition. In the offline training stage, we propose a threshold matrix and rate thresholds. Each element of the threshold matrix describes the minimal probability when a segment belongs to a sign, and rate thresholds are defined as the average probability for signs. So, if certain segment's evaluation is smaller than all the thresholds, it is regarded as a transitional movement and then it should be removed. In the online recognition stage, coarse segmentation, based on the threshold matrix, records the time interval for fine segmentation, and fine segmentation, based on Dynamic Time Warping(DTW) and Length-Root method, determines the endpoint for each candidate sign and selects the most possible one. The final recognition is obtained by concatenating the most possible signs. We evaluate our approach with Kinect-based dataset and the experiments demonstrate the effectiveness of our approach.