thoughtworks/arithmetic-sorl-data
Arithmetic SoRL Data Training and evaluation data for the SoRL Arithmetic Interpretability Study. Small transformers trained on integer addition/subtraction, with SoRL to externalize carry/borrow circuits as explicit abstraction tokens. Reference: Quirke et al., "Understanding Addition and Subtraction in Transformers" (2024). Paper: arXiv:2402.02619 — see Table 8 for complexity classification and Section 3 for sub-task definitions. Dataset Structure… See the full description on the dataset page: https://huggingface.co/datasets/thoughtworks/arithmetic-sorl-data.
add modular/test_seed42.json
add modular/train_seed42.json
add modular/config.json
Upload fixed_train/train_150K_seed42_meta.json with huggingface_hub
Upload fixed_train/train_150K_seed42.pt with huggingface_hub
Upload fixed_train/train_75K_seed42_meta.json with huggingface_hub
Upload fixed_train/train_75K_seed42.pt with huggingface_hub
Add fixed val set (seed=123, disjoint from train)
Fixed val set: 5K examples, seed=123, disjoint from train sets
Update data catalog: v2 fixed train sets (natural Quirke enrichment, no forced hard)
Fixed train 100K: Quirke enrichment, seed=42, natural hard ratio
Fixed train 100K: Quirke enrichment, seed=42, natural hard ratio
Fixed train 50K: Quirke enrichment, seed=42, natural hard ratio
Fixed train 50K: Quirke enrichment, seed=42, natural hard ratio
Fixed train 25K: Quirke enrichment, seed=42, natural hard ratio
Fixed train 25K: Quirke enrichment, seed=42, natural hard ratio
Fixed train 10K: Quirke enrichment, seed=42, natural hard ratio
Fixed train 10K: Quirke enrichment, seed=42, natural hard ratio
Metadata for 100K fixed train set: split distributions
Metadata for 50K fixed train set: split distributions
Metadata for 25K fixed train set: split distributions
Metadata for 10K fixed train set: split distributions
Fixed train set 100K: seed=42, 10% hard examples, Quirke enrichment
Fixed train set 50K: seed=42, 10% hard examples, Quirke enrichment
Fixed train set 25K: seed=42, 10% hard examples, Quirke enrichment
Fixed train set 10K: seed=42, 10% hard examples, Quirke enrichment
Fixed training set: 100K examples, seed=42, add_sub enriched
Fixed training set: 50K examples, seed=42, add_sub enriched
Fixed training set: 25K examples, seed=42, add_sub enriched
Fixed training set: 10K examples, seed=42, add_sub enriched
Initial data catalog: 10 entries
Upload eval_sets/eval_add_sub_6d_N25_seed42.json with huggingface_hub
Upload eval_sets/eval_add_sub_6d_N100_seed42.json with huggingface_hub
update dataset card: N=250 eval sets, Quirke arxiv link, subtraction enrichment, M6 note
Regenerate add_sub_6digit with borrow enrichment
Regenerate add_6digit with borrow enrichment
Delete add_sub_6digit for regeneration with borrow enrichment
Delete add_6digit for regeneration with borrow enrichment
Recreate sub_handcrafted with ST/SV columns
Recreate add_handcrafted with ST/SV columns
Recreate add_sub_6digit with ST/SV columns
Recreate add_6digit with ST/SV columns
Delete sub_handcrafted for regeneration
Delete add_handcrafted for regeneration
Delete add_sub_6digit for regeneration
Delete add_6digit for regeneration
Add handcrafted configs to dataset card
Add sub_handcrafted eval set (Quirke handcrafted)
Add add_handcrafted eval set (Quirke handcrafted)
Add dataset configs for HF viewer