datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
generalization-dynamics-evals
Generalization Dynamics — Main Eval Suite
Prepared test sets for the 6 main evaluation families from
Generalization dynamics across fine-tuning
(Table 1).
Use with the unified runner:
https://github.com/jiaxin-wen/FT-generalization/tree/main/release
from huggingface_hub import snapshot_download
root = snapshot_download(
repo_id="jiaxin-wen/generalization-dynamics-evals", repo_type="dataset")
Or browse a single task (the dataset viewer shows all configs):
from datasets… See the full description on the dataset page: https://huggingface.co/datasets/jiaxin-wen/generalization-dynamics-evals.compositional-generalization-benchmark
Compositional Generalization Benchmark (CGB)
Benchmark accompanying "Beyond Benchmark Illusions: A Diagnostic Framework
for Compositional Generalization in LLM Mathematical Reasoning."
Overview
CGB tests whether LLM math reasoning generalizes across three types of
compositional perturbation applied to GSM8K problems: numerical
perturbation, structural reformulation, and clause injection. The
benchmark contains 1168 problems (300 source + 868 variants), evaluated… See the full description on the dataset page: https://huggingface.co/datasets/monanem/compositional-generalization-benchmark.