Datasetpublic

Amartya77/Arithmetic

Recursive Arithmetic Transformer training frames There was no static training file. The model sampled integers online every step. This dataset replays that sampler with the same Python RNGs used in train_recursive: RNG Seed What it draws mix_rng seed + 91 task mix, operand lengths, integers offset_rng seed + 17 Position Coupling origin seed = 42 for every run. Each train() call resets both RNGs, so later finetunes are not a continuation of earlier streams.… See the full description on the dataset page: https://huggingface.co/datasets/Amartya77/Arithmetic.

sourceHugging Facemitupdated 1mo agoView on Hugging Face

0likes13downloads

Dataset Card

Recursive Arithmetic Transformer training frames

There was no static training file. The model sampled integers online every step. This dataset replays that sampler with the same Python RNGs used in train_recursive:

RNG	Seed	What it draws
`mix_rng`	`seed + 91`	task mix, operand lengths, integers
`offset_rng`	`seed + 17`	Position Coupling origin

seed = 42 for every run. Each train() call resets both RNGs, so later finetunes are not a continuation of earlier streams.

Code: Amartya-Roy/recursive-arithmetic-transformer

Regenerate locally:

bash

python export_training_data.py --output data/hf

What a row is

One frame the transformer saw, not a full n-digit multiply.

Digits are least-significant first. Only completion is trained (label_mask).

`task`	Supervised target	Operands
`base`	scratchpad + 1×1 / 2×2 product	lengths 1–2
`split`	`SPLIT`, `k`, four copied half-pairs	lengths 3–12 (15% up to 32)
`combine`	digits of `P_ll + (P_lh+P_hl)·10^k + P_hh·10^{2k}`	templated `COMBINE` prompt

The unique multiply curriculum is only 1- and 2-digit pairs. Longer digit strings appear as copy (split) or add (combine).

Example (`base`)

<bos> 3 2 * 7 4 = 1 4 1 + 8 6 1 = 1 8 0 1 <eos>

23 × 47 = 1081. First supervised token is a digit, never SPLIT.

Example (`split`, learned router)

<bos> 4 3 2 1 * 8 7 6 5 = SPLIT 2 CALL 1 4 3 | 8 7 CALL 2 4 3 | 6 5 CALL 3 2 1 | 8 7 CALL 4 2 1 | 6 5 <eos>

1234 × 5678, k=2. The net copies halves; it does not emit the 8-digit product.

Example (`combine`)

<bos> COMBINE 2 | 2 5 6 2 | 4 0 9 1 | 6 3 9 | 2 7 6 = 2 5 6 6 0 0 7 <eos>

7006652 LSD-first. Place IDs (column position_ids) encode the 10^k / 10^{2k} shifts.

Configs (sequential trainings)

Config	Steps × batch	Mix base/split/combine	`SPLIT` in	Bit-identical to the run?
`scratch_pretrain`	15k × 64	0.40 / 0.30 / 0.30	prompt	no — current sampler replay
`combine_finetune`	10k × 64	0.10 / 0.10 / 0.80	prompt	no
`combine_k2_finetune`	10k × 64	0.10 / 0.10 / 0.80	prompt	no
`allk_finetune`	10k × 64	0.15 / 0.10 / 0.75	prompt	no — last length-gated ckpt
`router_finetune`	10k × 64	0.25 / 0.40 / 0.35	target	yes — `recursive_block_router.pt`

router_finetune is the dump that matches checkpoints/recursive_block_router.pt (8×8 exact 977/1000). split_train_max=12. First supervised token on split frames is SPLIT.

Earlier configs used a length-gated controller (SPLIT already in the prompt). Those dumps are replayed with the current sampler. Combine sampling evolved during development, so those rows show the recipe, not a guaranteed byte match to the original GPU stream.

Columns

Operands and products are strings so JSON does not round large integers.

Column	Meaning
`run`	which train() call
`step`, `index_in_batch`, `global_index`	where in the 10k/15k stream
`task`	`base` / `split` / `combine`
`left`, `right`, `length`	multiply operands (0 on combine)
`k`, `r1`–`r4`	combine parts (`k` also set on split)
`target`	product, combine sum, or `SPLIT k a\	b ...`
`text`	full token string
`prompt` / `completion`	unsupervised prefix vs trained suffix
`first_supervised_token`	`SPLIT` or a digit
`position_offset`, `position_ids`	Place-value IDs
`label_mask`	`1` = trained token
`exact_replay`	true only for `router_finetune`

License

MIT, same as the code repo.