The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers <s> Datasets <s> This paper presents a novel approach to learning to solve simple arithmetic word problems. Our system, ARIS, analyzes each of the sentences in the problem statement to identify the relevant variables and their values. ARIS then maps this information into an equation that represents the problem, and enables its (trivial) solution as shown in Figure 1. The paper analyzes the arithmetic-word problems “genre”, identifying seven categories of verbs used in such problems. ARIS learns to categorize verbs with 81.2% accuracy, and is able to solve 77.7% of the problems in a corpus of standard primary school test questions. We report the first learning results on this task without reliance on predefined templates and make our data publicly available. 1 <s> BIB001 </s> The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers <s> Datasets <s> This paper formalizes the problem of solving multi-sentence algebraic word problems as that of generating and scoring equation trees. We use integer linear programming to generate equation trees and score their likelihood by learning local and global discriminative models. These models are trained on a small set of word problems and their answers, without any manual annotation, in order to choose the equation that best matches the problem text. We refer to the overall system as ALGES. We compare ALGES with previous work and show that it covers the full gamut of arithmetic operations whereas Hosseini et al. (2014) only handle addition and subtraction. In addition, ALGES overcomes the brittleness of the Kushman et al. (2014) approach on single-equation problems, yielding a 15% to 50% reduction in error. <s> BIB002 </s> The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers <s> Datasets <s> This paper presents a novel approach to automatically solving arithmetic word problems. This is the first algorithmic approach that can handle arithmetic problems with multiple steps and operations, without depending on additional annotations or predefined templates. We develop a theory for expression trees that can be used to represent and evaluate the target arithmetic expressions; we use it to uniquely decompose the target arithmetic problem to multiple classification problems; we then compose an expression tree, combining these with world knowledge through a constrained inference framework. Our classifiers gain from the use of quantity schemas that supports better extraction of features. Experimental results show that our method outperforms existing systems, achieving state of the art performance on benchmark datasets of arithmetic word problems. <s> BIB003 </s> The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers <s> Datasets <s> Math word problems provide a natural abstraction to a range of natural language understanding problems that involve reasoning about quantities, such as interpreting election results, news about casualties, and the financial section of a newspaper. Units associated with the quantities often provide information that is essential to support this reasoning. This paper proposes a principled way to capture and reason about units and shows how it can benefit an arithmetic word problem solver. This paper presents the concept of Unit Dependency Graphs (UDGs), which provides a compact representation of the dependencies between units of numbers mentioned in a given problem. Inducing the UDG alleviates the brittleness of the unit extraction system and allows for a natural way to leverage domain knowledge about unit compatibility, for word problem solving. We introduce a decomposed model for inducing UDGs with minimal additional annotations, and use it to augment the expressions used in the arithmetic word problem solver of (Roy and Roth 2015) via a constrained inference framework. We show that introduction of UDGs reduces the error of the solver by over 10 %, surpassing all existing systems for solving arithmetic word problems. In addition, it also makes the system more robust to adaptation to new vocabulary and equation forms . <s> BIB004
There have been a number of datasets collected for the arithmetic word problems. We present their descriptions in the following and summarize the statistics of the datasets in Table 1. 1) AI2 BIB001 . There are 395 single-step or multi-step arithmetic word problems for the third, fourth, and fifth graders. It involves problems that can be solved with only addition and subtraction. The dataset is harvested from two websites: math-aids.com and ixl.com and comprises three subsets: MA1 (from math-aids.com), IXL (from ixl.com) and MA2 (from math-aids.com). Among them, IXL and MA2 are more challenging than MA1 because IXL contains more information gaps and MA2 includes more irrelevant information in its math problems. 2) IL BIB003 . The problems are collected from websites k5learning.com and dadsworksheets.com. The problems that require background knowledge (e.g., "apple is fruit" and "a week comprises 7 days") are pruned. To improve the diversity, the problems are clustered by textual similarity. For each cluster, at most 5 problems are retained. Finally, the dataset contains 562 single-step word problems with only one operator, including addition, subtraction, multiplication, and division. 3) CC BIB003 . The dataset is designed for mult-step math problems. It contains 600 multi-step problems without irrelevant quantities, harvested from commoncoresheets.com. The dataset involves various combinations of four basic operators, including (a) addition followed by subtraction; (b) subtraction followed by addition; (c) addition and multiplication; (d) addition and division; (e) subtraction and multiplication; and (f) subtraction and division. It is worth noting that this dataset does not incorporate irrelevant quantities in the problem text. Hence, there is no need to apply the quantity relevance classifier for the algorithms containing this component. 4) SingleEQ BIB002 . The dataset contains both single-step and multi-step arithmetic problems and is a mixture of problems from a number of sources, including math-aids.com, k5learning.com, ixl.com and a subset of the data from AI2. Each problem involves operators of multiplication, division, subtraction, and addition over non-negative rational numbers. 5) AllArith BIB004 . The dataset is a mixture of the data from AI2, IL, CC and SingleEQ. All mentions of quantities are normalized into digit representation. To capture how well the automatic solvers can distinguish between different problem types, near-duplicate problems (with over 80% match of unigrams and bigrams) are removed. Finally, there remain 831 math problems. 6) Dolphin-S. This is a subset of Dolphin18K  which originally contains 18, 460 problems and 5, 871 templates with one or multiple equations. The problems whose template is associated with only one problem are extracted as the dataset of Dolphin-S. It contains 115 problems with single operator and 6, 955 problems with multiple operators. 7) Math23K  . The dataset contains Chinese math word problems for elementary school students and is crawled from multiple online education websites. Initially, 60, 000 problems with only one unknown variable are collected. The equation templates are extracted in a rule-based manner. To ensure high precision, a large number of problems that do not fit the rules are discarded. Finally, 23, 161 math problems with 2, 187 templates are remained.