datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
tiny-supervised-datasetTinyStoriesV2Data is from https://huggingface.co/datasets/roneneldan/TinyStories/:
TinyStoriesV2-GPT4-train.txt - Is a new version of the dataset that is based on generations by GPT-4 only (the original dataset also has generations by GPT-3.5 which are of lesser quality). It contains all the examples in TinyStories.txt which were GPT-4 generated as a subset (but is significantly larger).
This dataset was used to train https://github.com/noanabeshima/tiny_model/.
The data was preprocessed with:
from… See the full description on the dataset page: https://huggingface.co/datasets/noanabeshima/TinyStoriesV2.somali-tinystoriestinystories-icr-data-v2tiny_qa_benchmark_pp
Tiny QA Benchmark++ (TQB++)
Tiny QA Benchmark++ (TQB++) is an ultra-lightweight evaluation suite designed to expose critical failures in Large Language Model (LLM) systems within seconds. It serves as the LLM analogue of software unit tests, ideal for rapid CI/CD checks, prompt engineering, and continuous quality assurance in modern LLMOps.
This Hugging Face dataset repository hosts the core English dataset and various synthetically generated multilingual and topical dataset packs… See the full description on the dataset page: https://huggingface.co/datasets/vincentkoc/tiny_qa_benchmark_pp.TinyTextThe entire NanoPhi Dataset is at train.jsonl
Separate Tasks Include
Math (Metamath, mammoth)
Code (Code Search Net)
Logic (Open-platypus)
Roleplay (PIPPA, RoleplayIO)
Textbooks (Tiny-text, Sciphi)
Textbook QA (Orca-text, Tiny-webtext)
tinystories_dataset_arabicTinyBooks-QA-Chinese本数据集已停止更新，请移步https://huggingface.co/datasets/fzmnm/TinyStoriesAdv-zh
TinyBooks-QA-Chinese
Inspired by the (TinyStories)[https://arxiv.org/abs/2305.07759] paper, where a small language model exhibits strong capabilities when trained on high-quality, 🍼baby-friendly stories synthesized by AI, I present an AI-generated Encyclopedia suitable for kindergarten and grade school levels.
This AI-synthesized dataset converts classical literature into a question-answer style curriculum with… See the full description on the dataset page: https://huggingface.co/datasets/fzmnm/TinyBooks-QA-Chinese.TinyStories-Multilingual
Novelist: TinyStories Multilingual Edition
Dataset Summary
The TinyStories Multilingual Edition is a high-fidelity synthetic dataset of short, child-safe fiction designed to stress-test literary consistency, emotional warmth, and multilingual fluency in small models. Derived from the broader Novelist ecosystem, this subset focuses on narrative simplicity paired with complex moral and social themes.
The dataset contains 15,688 high-quality stories across 28 languages. Each… See the full description on the dataset page: https://huggingface.co/datasets/Dxniz/TinyStories-Multilingual.TinyMCQReasoningtiny-random-model-summaryglm-moe-dsa-tiny-cpu-repro-v1
Tiny GLM MoE DSA: two CPU captures, forced zero-KL replay
Reproducibility evidence for
malaiwah/glm-moe-dsa-tiny-random-bf16,
checkpoint/config/tokenizer revision 45563636ef723acfb826755493447dc40c7a0c37.
This is a synthetic pipeline test, not a quality benchmark, quantization measurement,
qualified production reference, or registry submission. The model is random-init.
No GPU or paid cloud job was used.
Observed result
Two fresh capture processes, two CPU… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/glm-moe-dsa-tiny-cpu-repro-v1.tinygiant-omni-test-featurestinybrain-instruct-sft-200k
TinyBrain Instruct 200K
A 196k+ row English SFT dataset for training tiny instruction-following language models.
TinyBrain Instruct 200K is a synthetic supervised fine-tuning dataset made for small language models, especially models around 100M–500M parameters.
The dataset focuses on short, clear, learnable assistant responses across education, basic math reasoning, clean conversation, planning, simplification, simple coding, and honesty/uncertainty behavior.
Most… See the full description on the dataset page: https://huggingface.co/datasets/exnivo/tinybrain-instruct-sft-200k.mit67_swin_tiny-oof-featuresTiny-Open-Domain-BooksA tiny example dataset consisting of four books dedicated to the open domain in JSONL format:
Alice in Wonderland - Lewis Caroll
Dracula - Bram Stoker
The Wonderful Wizard of Oz - L. Frank Baum
The Count of Monte Cristo - Alexandre Dumas & Auguste Maquet
All works are open domain, thus this dataset is also dedicated to the open domain.
The dataset has been made to have extremely long context lengths, ideally as close to 2048 at possible without cutting off chunks in strange places. Each… See the full description on the dataset page: https://huggingface.co/datasets/Blackroot/Tiny-Open-Domain-Books.TinyEncyclopedias-Chinese本数据集已停止更新，请移步https://huggingface.co/datasets/fzmnm/TinyStoriesAdv-zh
TinyEncyclopediasChinese
Inspired by the papers (TinyStories)[https://arxiv.org/abs/2305.07759] and (Textbooks Are All You Need)[https://arxiv.org/abs/2306.11644], where a small language model exhibits strong capabilities when trained on high-quality, kid-friendly stories synthesized by AI, I present an AI-generated Encyclopedia suitable for kindergarten and grade school levels.
This dataset follows my previous… See the full description on the dataset page: https://huggingface.co/datasets/fzmnm/TinyEncyclopedias-Chinese.tiny-instruct-kotinybrain-pretrain-corpus-2b
TinyBrain Pretrain Corpus 2B
A mixed-source English pretraining corpus for training small language models.
TinyBrain Pretrain Corpus 2B is a mixed-source dataset built for pretraining small causal language models, especially the TinyBrain-100M Base model.
The dataset combines educational text, factual/wiki-style text, math reasoning data, Python code-summary data, clean web text, and conversation-style data. It is designed to give small models a useful general foundation… See the full description on the dataset page: https://huggingface.co/datasets/exnivo/tinybrain-pretrain-corpus-2b.qwen3-5-tiny-fidelity-root-v1
qwen3_5 random CPU fixture root
A root fidelity dataset in hidden form, produced by engines/tools/hf_capture.py from malaiwah/qwen3-5-tiny-random-bf16.
The cut
the final hidden state handed to lm_head -- after the text model's final norm and immediately before the head matmul -- captured as the head module's input via torch.nn.Module.register_forward_pre_hook; replay applies the head ONLY (no final norm at replay time: the capture already sits after it). Same cut… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/qwen3-5-tiny-fidelity-root-v1.TinyStories-SFT
This is Finetuning dataset for tinystories
This is for finetuning model trained on dataset https://huggingface.co/datasets/roneneldan/TinyStories
json on this dataset
I'll update readme later
deepseek-v4-tiny-fidelity-root-v1
deepseek-v4 random CPU fixture root
A root fidelity dataset in hidden form, produced by engines/tools/hf_capture.py from malaiwah/deepseek-v4-tiny-random-bf16.
The cut
the final hidden state handed to lm_head -- after the text model's final norm and immediately before the head matmul -- captured as the head module's input via torch.nn.Module.register_forward_pre_hook; replay applies the head ONLY (no final norm at replay time: the capture already sits after it).… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/deepseek-v4-tiny-fidelity-root-v1.qwen3-5-gguf-tiny-fidelity-root-v1
qwen35-gguf random CPU fixture root
A root fidelity dataset in hidden form, produced by engines/tools/hf_capture.py from malaiwah/qwen3-5-gguf-tiny-random-bf16.
The cut
the final hidden state handed to lm_head -- after the text model's final norm and immediately before the head matmul -- captured as the head module's input via torch.nn.Module.register_forward_pre_hook; replay applies the head ONLY (no final norm at replay time: the capture already sits after it).… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/qwen3-5-gguf-tiny-fidelity-root-v1.TinyChartDataqwen4-exp-tiny-fidelity-root-v1
qwen4_exp random CPU fixture root
A root fidelity dataset in hidden form, produced by engines/tools/hf_capture.py from malaiwah/qwen4-exp-tiny-random-bf16.
The cut
the final hidden state handed to lm_head -- after the text model's final norm and immediately before the head matmul -- captured as the head module's input via torch.nn.Module.register_forward_pre_hook; replay applies the head ONLY (no final norm at replay time: the capture already sits after it). Same… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/qwen4-exp-tiny-fidelity-root-v1.tiny-llm-synthetic-qa
Tiny-LLM: Synthetic Question-Answering Dataset
Dataset Description
This dataset was created for the fine-tuning stage of the Tiny-LLM Project, a project focused on training and evaluating compact language models from scratch.
It contains 706,727 high-quality, synthetic multi-turn Question-Answering (Q&A) conversations in English, generated using the Gemini API. The dataset was designed to teach small models instruction-following capabilities across a diverse range of… See the full description on the dataset page: https://huggingface.co/datasets/Gabriel8/tiny-llm-synthetic-qa.k2-horizon-tiny-fidelity-root-v1
k2-horizon random CPU fixture root
A root fidelity dataset in hidden form, produced by engines/tools/hf_capture.py from malaiwah/k2-horizon-tiny-random-bf16.
The cut
the final hidden state handed to lm_head -- after the text model's final norm and immediately before the head matmul -- captured as the head module's input via torch.nn.Module.register_forward_pre_hook; replay applies the head ONLY (no final norm at replay time: the capture already sits after it).… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/k2-horizon-tiny-fidelity-root-v1.TinyMathStories_gpt-oss-20b
TinyMathStories
A TinyStories-style corpus extended with math and lightweight reasoning.
This dataset keeps the child-level vocabulary and short narrative style of TinyStories (Microsoft Research, Eldan & Li, 2023) and mixes in basic numeracy (counting, addition/subtraction, simple equations, fractions, measurement) and short justifications—so tiny models can practice coherent English and early math/logic.
Research, generation, and curation by AlgoDriveAI.Inspired by and… See the full description on the dataset page: https://huggingface.co/datasets/AlgoDriveAI/TinyMathStories_gpt-oss-20b.minimax-m2-tiny-fidelity-root-v1
minimax-m2 random CPU fixture root
A root fidelity dataset in hidden form, produced by engines/tools/hf_capture.py from malaiwah/minimax-m2-tiny-random-bf16.
The cut
the final hidden state handed to lm_head -- after the text model's final norm and immediately before the head matmul -- captured as the head module's input via torch.nn.Module.register_forward_pre_hook; replay applies the head ONLY (no final norm at replay time: the capture already sits after it).… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/minimax-m2-tiny-fidelity-root-v1.glm5-next-tiny-fidelity-root-v1
glm5_next random CPU fixture root
A root fidelity dataset in hidden form, produced by engines/tools/hf_capture.py from malaiwah/glm5-next-tiny-random-bf16.
The cut
the final hidden state handed to lm_head -- after the text model's final norm and immediately before the head matmul -- captured as the head module's input via torch.nn.Module.register_forward_pre_hook; replay applies the head ONLY (no final norm at replay time: the capture already sits after it). Same… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/glm5-next-tiny-fidelity-root-v1.