Emulated-Inc/competition-answer-math-training-pool

Competition answer mathematics training pool

Competition mathematics problems that ask for a single final answer, three public datasets gathered at pinned revisions, shipped twice over. sources/ holds each dataset the way its publisher ships it, in its own file format with its own fields and nothing renamed, 113045 rows across three folders. pool/ holds the union of those same datasets in one format, one JSON object per line, deduplicated by problem text and reduced to 107637 rows, every row labelled with the dataset it came from, that dataset's revision and its provenance class.

Nothing here is filtered for quality or for difficulty. Problems written by people and problems copied from competition papers sit side by side, labelled, and which of them to use is the reader's decision. Take the union if one format is what you want, take a source folder if you already know that dataset and want its own columns.

Layout

README.md
pool/pool-00000-of-00003.jsonl  ...  pool-00002-of-00003.jsonl   the union, 107637 rows
sources/deepscaler/              1 file,   38514 rows
sources/omni_math/               1 file,    4251 rows
sources/numinamath_1_5/          3 files,  70280 rows

The union is shuffled with a fixed seed, so any prefix of any file is a sample of the whole pool rather than of one source. The pool is about 242 MB on disk, 168 MB of it the union and 74 MB the source folders. One of the three files in the last folder holds no rows at all, because neither of the two subsets kept from that dataset appears in that shard.

The union under pool/

One JSON object per line, with these fields.

Counts over the 107637 rows: 60971 carry an answer and 45625 carry a solution. Of the answers, 60939 are the source's own answer field and 32 were read out of the last box of a solution, and 30239 of them are a plain integer, the rest being closed forms such as fractions, radicals and expressions. By provenance of the problem: 106745 collected, 892 human, none model-generated. Everything is in English.

By source: 69251 rows from numinamath15, 37494 from deepscaler and 892 from omnimath. The last number is small because the union keeps one row per distinct problem and prefers the copy that carries the most, an answer first, then a solution, and then the earlier source in the list below. The deepscaler folder was itself assembled from competition archives and from the omnimath collection, so most of that collection's problems appear in it as well and are kept there, with the omni_math folder still holding them in full under sources/.

Roughly a third of the rows have neither an answer nor a solution. Those are the proof and construction problems of the forum subset, which state a claim to prove rather than a quantity to compute. If you want only the rows a checker can grade, filter on has_answer, and if you want only the rows whose answer is an integer, filter on that too.

The sources under sources/

deepscaler, agentica-org/DeepScaleR-Preview-Dataset at b6ae8c60f5c1f2b594e2140b91c49c9ad0949e29, 38514 rows, one JSON file holding a list of objects with problem, answer and solution, the answer being the final answer alone. MIT. Provenance collected, solutions collected. About forty thousand competition problems with a single short answer, compiled from the papers of the American Invitational Mathematics Examination of 1984 to 2023, the American Mathematics Competitions papers before 2023, the Omni-MATH collection and the STILL collection. The problems and the worked solutions are the ones the competitions and the archives published. A model was used only to read the final answer out of each solution, and answers a symbolic checker could not handle were dropped. Most of its rows carry an answer and no solution text.

omni_math, KbsdJames/Omni-MATH at 40ba231d8f16e29ecd40e6407e2c8640145a8f62, 4251 rows, one JSON object per line with problem, solution, answer, domain, difficulty and source, the source naming the competition. Apache 2.0. Provenance human, solutions human. Olympiad problems collected from the official competition websites and the Art of Problem Solving wiki, with the solutions those sites published and a difficulty rating and a subject path added by hand. Problems and solutions are both written by people. Its source column names more than eighty competitions, the largest of them the International Mathematical Olympiad shortlists, the Harvard MIT Mathematics Tournament, the United States of America Mathematical Olympiad and the Putnam competition.

numinamath_1_5, AI-MO/NuminaMath-1.5 at 1b05109f9e5c1ad06c0663519502416c30b300f8, 70280 rows, parquet shards with problem, solution, answer, problemtype, questiontype, problemisvalid, solutionisvalid, source and synthetic. An answer of proof or notfound means the row has no short answer. Apache 2.0. Provenance collected, solutions model-generated. Two subsets of the largest compilation of competition mathematics on the hub, kept and the rest left out: aops_forum, the problems posted to the Art of Problem Solving forum, 64183 rows of the union, and amc_aime, the problems of the American Mathematics Competitions and American Invitational Mathematics Examination archives, 5068 rows of the union. Both are problems people wrote and posted, gathered by crawling. The solutions were rewritten into a uniform style by a large language model whatever the problem's origin, so the solution text is model-generated across the whole file, and the short answers were extracted by the same model. The compilation's school mathematics, its Chinese curriculum material and its model-written problems are not here: this pool is competition mathematics with a short answer.

Provenance in plain words

• —human: a person wrote the text, with no model in the loop.
• —collected: the text is a real artefact of people doing something, a competition paper, an examination, a forum post, gathered rather than written for a dataset.
• —model-generated: a language model wrote it. No problem in this pool is in this class. Every solution of the largest folder is, because that compilation rewrote its solutions with a model.
• —rule-generated: a program or a template produced it. Nothing in this pool is in this class.

Almost all of the pool is collected: real competition problems, gathered at scale. If you care that the reasoning was written by a person rather than a model, filter solution_provenance to human and collected, which leaves the omni_math rows and the deepscaler rows that carry a solution.

Licences

Each source is redistributed here under the licence its publisher declares for it, and the licence of every row is the licence of its source folder.

Apache 2.0 is the more demanding of the two, so it governs the pool as a whole. Nothing here is licensed for non-commercial use and nothing here is untagged. The compilations gather material from many places, and the licence above is the one their publisher applies to the compilation.

Several large corpora that would otherwise belong here are deliberately absent. Some competition archives exist on the hub only under a non-commercial share-alike licence and cannot be redistributed at all, so the omni_math folder is what carries the problems of those competitions that it happens to hold. The MATH training corpus is not included: the original upload was disabled on a takedown notice from the rightsholder, which also asked for datasets branched off it to be removed, and of the mirrors that remain one declares no licence at all. The Art of Problem Solving instruction corpus is not included either: its authors disclaim the licence tag the upload carries, and the forum it was crawled from now publishes a machine-readable reservation of rights against training use.

A note on the counts

The counts above are the counts in this pool. Rows were removed from every source while it was assembled, so they do not match the totals the source repositories publish.