datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
mod-arithmetic
Modular Arithmetic Dataset
Synthetic dataset of modular-arithmetic problems of the form a mod b,
paired with the result and a hypothesis about the most suitable tokenizer.
Tokenizer hypothesis
For a mod b where b = 2^k × 5^j (no other prime factors), only the
rightmost max(k, j) digits of a determine the answer, because
10^max(k,j) ≡ 0 (mod b). A tokenizer that groups digits right-to-left
in chunks of that size exposes the relevant information as a single token.
For all… See the full description on the dataset page: https://huggingface.co/datasets/flexitok/mod-arithmetic.multilingual-addition
Multilingual Addition Dataset
Synthetic dataset of addition problems of the form a+b=answer, where a
and b are written-form representations of integers in 21 languages, plus
a 22nd split using raw digit strings.
Task format
Each sample contains:
field
type
description
a_str
str
written-form (or digit) representation of a
a_digit
int
integer value of a
b_str
str
written-form (or digit) representation of b
b_digit
int
integer value of b
answer
str… See the full description on the dataset page: https://huggingface.co/datasets/flexitok/multilingual-addition.