license:apache-2.0 arxiv:2305.15017 arxiv:1905.13319 region:us
---
license: apache-2.0
---

# Dataset Card for "Calc-math_qa"


## Summary

This dataset is an instance of math_qa dataset, converted to a simple HTML-like language that can be easily parsed (e.g. by BeautifulSoup). The data contains 3 types of tags:
- gadget: A tag whose content is intended to be evaluated by calling an external tool (sympy-based calculator in this case)
- output: An output of the external tool
- result: The final answer of the mathematical problem (a number)


## Supported Tasks

The dataset is intended for training Chain-of-Thought reasoning **models able to use external tools** to enhance the factuality of their responses.
This dataset presents in-context scenarios where models can outsource the computations in the reasoning chain to a calculator.


## Construction Process

We took the original math_qa dataset, parsed the nested formulas, linearized them into a sequence (chain) of operations, and replace all advanced
function calls (such as `circle_area`) with explicit elementary operations. We evaluate all the steps in each example and filter out examples if their
evaluation does not match the answer selected as correct in the data with a 5% tolerance. The sequence of steps is then saved in HTML-like language
in the `chain` column. We keep the original columns in the dataset for convenience.

You can read more information about this process in our [technical report](https://arxiv.org/abs/2305.15017).


## Content and Data splits

Content and splits correspond to the original math_qa dataset.
See [mathqa HF dataset](https://huggingface.co/datasets/math_qa) and [official website](https://math-qa.github.io/) for more info.

Columns:

- `question` - th description of a mathematical problem in natural language
- `chain` - Solution in the form of step-by-step calculations encoded in simple html-like language. computed from `annotated_formula` column
- `result` - the result of the problem as string (can be integer, floating number, fraction, ...)
- `result_float` - the result converted to a float
- `options` - dictionary with choices 'a' to 'e' as possible solutions
- `options_num` - same as 'options', but with simple parsing to extract the number from string. This is best-effort only - not all values are (or can be) extracted correctly
- `correct_option` - correct options, one of 'a', ..., 'e', should match with `result`
- `rationale` - human-annotated free-text reasoning that leads to the correct answer
- `annotated_formula` - human-annotated nested expression that (approximately) evaluates to the selected correct answer
- `linear_formula` - same as `annotated_formula`, but linearized. Provided by original math_qa authors
- `index` - index of the example in the original math_qa dataset




## Licence

Apache 2.0, consistently with the original dataset.


## Cite

If you use this version of dataset in research, please cite the [original MathQA paper](https://arxiv.org/abs/1905.13319), and also [our technical report](https://arxiv.org/abs/2305.15017) as follows:

```bibtex
@article{kadlcik2023calcx,
         title={Calc-X: Enriching Arithmetical Chain-of-Thoughts Datasets by Interaction with Symbolic Systems}, 
         author={Marek Kadlčík and Michal Štefánik},
         year={2023},
         eprint={2305.15017},
         archivePrefix={arXiv},
         primaryClass={cs.LG}
}
```