Mathematical reasoning is often used to measure the reasoning power of a model. Early benchmarks contain simple arithmetic operations Hosseini et al. (2014); Koncel-Kedziorski et al. (2015); Roy and Roth (2015); Koncel-Kedziorski et al. (2016). Ling et al. (2017) labels the reasoning process in natural language form, and Amini et al. (2019) builds on AQUA by labeling the reasoning process in program form. Later benchmarks Miao et al. (2020); Patel et al. (2021); Cobbe et al. (2021); Gao et al. (2023) contain more complex and diverse questions. Zhu et al. (2021); Chen et al. (2021); Chen et al. (2022) require reasoning based on the table content. There are also general benchmarks Hendrycks et al. (2021); Mishra et al. (2022); Baa et al. (2022) and reading comprehension form benchmarks Dua et al. (2019); Chen et al. (2023). Recently, Yu et al. (2021) endowed pre-trained model with the ability of mathematical reasoning by using hierarchical reasoning and knowledge.