littlelearner/LittleCurriculum-Chat

1---2license: odc-by3task_categories:4  - text-generation5  - question-answering6language:7  - en8tags:9  - synthetic10  - sft11  - instruction-tuning12  - education13size_categories:14  - 100K<n<1M15configs:16  - config_name: math17    data_files: math.parquet18  - config_name: general_knowledge19    data_files: general_knowledge.parquet20---21 22# LittleCurriculum-Chat23 24Synthetic K–5 chat data generated with Gemini 2.5 Flash and filtered with the25[LittleCurriculum filter](https://github.com/littlelearner-ll/littlecurriculum-filter).26Used to train the [LittleLearner](https://huggingface.co/littlelearner) models.27 28| Config | Rows | Seeded from | Content |29|---|---|---|---|30| `math` | 79,543 | MegaMath-Web-Pro-Max | Word problems with step-by-step worked solutions |31| `general_knowledge` | 484,787 | [LittleCurriculum](https://huggingface.co/datasets/littlelearner/LittleCurriculum) | Reading comprehension, factual QA, explanation, summarisation, definitions |32 33Seeds are real documents rather than topic prompts, which keeps the topic34distribution broad. The seed documents are not included here, so some questions35assume context the reader does not have ("what is the main idea of this36article?"). `seed_id` is the FineWeb-Edu document id, so the source text can be37recovered by joining against38[LittleCurriculum](https://huggingface.co/datasets/littlelearner/LittleCurriculum).39 40Every row carries a `messages` column in `[{role, content}]` form alongside the41structured fields.42 43```python44from datasets import load_dataset45ds = load_dataset("littlelearner/LittleCurriculum-Chat", "general_knowledge")46```47 48## Related data49 50For instruction fine-tuning we additionally used SmolTalk, MMLU51auxiliary-train and GSM8K, each filtered with our pipeline. For GRPO we used52grade-stratified synthetic problems from the MathCAMPS pipeline alongside53GSM8K, again filtered with our pipeline. We do not redistribute these — they54derive from public datasets and reproduce in one command:55 56```bash57python filter_k5.py --hf-dataset HuggingFaceTB/smol-smoltalk --out smoltalk_k5.parquet58```59 60In our runs this data was most useful mixed into pretraining and midtraining61rather than reserved for a dedicated SFT stage.62 63## License64 65ODC-By 1.0, inherited from FineWeb-Edu via LittleCurriculum. Generated with66Google Gemini.67 68## Citation69 70```bibtex71@misc{li2026littlelearner,72      title={LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure},73      author={Fanfei Li and Jana Zeller and Manuel Prada-Corral and Thaddäus Wiedemer and Prasanna Mayilvahanan and Ryan Cotterell and Wieland Brendel},74      year={2026},75      eprint={2608.13545},76      archivePrefix={arXiv},77      primaryClass={cs.CL},78      url={https://arxiv.org/abs/2608.13545},79}80```81