MU-NLPC/Calc-asdiv_a

Dataset Card for Calc-asdiv_a

Summary

The dataset is a collection of simple math word problems focused on arithmetics. It is derived from the arithmetic subset of ASDiv (original repo).

The main addition in this dataset variant is the chain column. It was created by converting the solution to a simple html-like language that can be easily parsed (e.g. by BeautifulSoup). The data contains 3 types of tags:

• —gadget: A tag whose content is intended to be evaluated by calling an external tool (sympy-based calculator in this case)
• —output: An output of the external tool
• —result: The final answer to the mathematical problem (a number)

Supported Tasks

This variant of the dataset is intended for training Chain-of-Thought reasoning models able to use external tools to enhance the factuality of their responses. This dataset presents in-context scenarios where models can outsource the computations in the reasoning chain to a calculator.

Data splits

The dataset does not contain data splits. We consider the whole dataset as a testing benchmark.

Attributes:

• —id: id of the example
• —question problem description in English
• —chain: series of simple operations (derived from expression) that lead to the solution
• —result: the solution for x as a number or fraction (string)
• —result_float: same as result but converted to a float
• —result_unit: the units of the result
• —grade: an estimate of the school grade in which the problem would be practiced
• —source_question: the source from which the example originates

Attributes id, question, chain, and result are present in all datasets in the Calc-X collection.

Related work

This dataset was created as a part of a larger effort in training models capable of using a calculator during inference, which we call Calcformers.

• —**Calc-X collection** - datasets for training Calcformers
• —**Calcformers collection** - calculator-using models we trained and published on HF
• —**Calc-X and Calcformers paper**
• —**Calc-X and Calcformers repo**

Here are links to the original dataset:

• —**original ASDiv dataset and repo**
• —**original ASDiv paper**

Licence

CC BY-NC 4.0, consistent with the original source dataset linked above.

Cite

If you use this dataset in research, please cite the original ASDiv paper, and Calc-X collection as follows:

@inproceedings{kadlcik-etal-2023-soft,
    title = "Calc-X and Calcformers: Empowering Arithmetical Chain-of-Thought through Interaction with Symbolic Systems",
    author = "Marek Kadlčík and Michal Štefánik and Ondřej Sotolář and Vlastimil Martinek",
    booktitle = "Proceedings of the The 2023 Conference on Empirical Methods in Natural Language Processing: Main track",
    month = dec,
    year = "2023",
    address = "Singapore, Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/2305.15017",
}