We propose a new framework for the recognition of online handwritten graphics. Three main features of the framework are its ability to treat symbol and structural level information in an integrated way, its flexibility with respect to different families of graphics, and means to control the tradeoff between recognition effectiveness and computational cost. We model a graphic as a labeled graph generated from a graph grammar. Non-terminal vertices represent subcomponents, terminal vertices represent symbols, and edges represent relations between subcomponents or symbols. We then model the recognition problem as a graph parsing problem: given an input stroke set, we search for a parse tree that represents the best interpretation of the input. Our graph parsing algorithm generates multiple interpretations (consistent with the grammar) and then we extract an optimal interpretation according to a cost function that takes into consideration the likelihood scores of symbols and structures. The parsing algorithm consists in recursively partitioning the stroke set according to structures defined in the grammar and it does not impose constraints present in some previous works (e.g. stroke ordering). By avoiding such constraints and thanks to the powerful representativeness of graphs, our approach can be adapted to the recognition of different graphic notations. We show applications to the recognition of mathematical expressions and flowcharts. Experimentation shows that our method obtains state-of-the-art accuracy in both applications.
To cope with the structural variance of diagrams, some approaches introduce strong constraints in the input, as requiring all symbols to have only one stroke @cite_11 , or loop-like symbols to be written by consecutive strokes @cite_38 . With respect to symbol recognition, detection of texts (or text box) and arrow symbols are regarded as more difficult, as they do not present a fixed shape. For instance, Carton @cite_12 determine box symbols (like , and structure) and then select the best interpretations using a deformation metric. Text symbols are recognized only after box symbols. Bresler @cite_6 also first recognize possible box and arrow symbols, and leave text recognition as a last step. After symbol candidates are identified, the best symbol combination is selected through a max-sum optimization process.
@cite_38 @cite_6 @cite_12 @cite_11
2013257774 2031948510 1603389798
In order to segment and recognize on-line handwritten flowchart symbols precisely, we propose a method that segments the graphic symbols based on the loop structure and recognize the segmented symbols by using SVMs. In our experiments, low error rate of 3.37 for symbol segmentation and high recognition rate of 97.6 were obtained. We also propose a beautification and editing method for recognized symbols, and implement them to construct a prototype system. We compare an input time for drawing flowcharts between our system and a traditional application using icon-based interface. As a result, the input time on our system was faster than that on traditional one for flowcharts without texts. We present our recent model of a diagram recognition engine. It extends our previous work which approaches the structural recognition as an optimization problem of choosing the best subset of symbol candidates. The main improvement is the integration of our own text separator into the pipeline to deal with text blocks occurring in diagrams. Second improvement is splitting the symbol candidates detection into two stages: uniform symbols detection and arrows detection. Text recognition is left for post processing when the diagram structure is already known. Training and testing of the engine was done on a freely available benchmark database of flowcharts. We correctly segmented and recognized 93.0 of the symbols having 55.1 of the diagrams recognized without any error. Considering correct stroke labeling, we achieved the precision of 95.7 . This result is superior to the state-of-the-art method with the precision of 92.4 . Additionally, we demonstrate the generality of the proposed method by adapting the system to finite automata domain and evaluating it on own database of such diagrams. The electronic white board and the tablet PC are bringing new technologies to modern education. This paper presents a pen-based flowchart recognition system for programming teaching, which uses hybrid SVM-HMM algorithm for sketch recognition. In this algorithm, ICA is used to reduce the dimension of features, a set of fuzzy SVMs are used as preliminary feature classifiers to produce fix length feature vector, which acts as a probability evaluator in the hidden states of Hidden Markov Models, and HMMs are employed as finally classifiers to recognize the unknown pattern. Experiment results show the hybrid algorithm has good learning and recognition ability. And based on this algorithm, an intelligent whiteboard system for programming teaching is designed to identify the sketches into the programming flowchart, and finally converts it into C language programs. User's evaluation shows it is natural for the teachers and the students with a flexible and effective interactive teaching pattern. Therefore, such system brings a new programming teaching patterns and help students to stride the obstacle between the flowchart and the programming language. Students can learn the abstract programming idea and the concrete coding skills effectively and efficiently by the visual comparative learning assisted by the intelligent whiteboard system.