Pondsiders/tinystories-gpt4-instruct
tinystories-gpt4-instruct Request→story pairs for supervised fine-tuning of small language models, derived from karpathy/tinystories-gpt4-clean. Each example pairs a natural-language request ("Can you tell me a story about a boy named Tim?") with a TinyStories story that satisfies it. The dataset lives on Hugging Face; the notebook that generates it lives on GitHub. This is not roneneldan/TinyStoriesInstruct. That dataset frames its tasks in a structured format (Words:… See the full description on the dataset page: https://huggingface.co/datasets/Pondsiders/tinystories-gpt4-instruct.
tinystories-gpt4-instruct
Request→story pairs for supervised fine-tuning of small language models, derived from karpathy/tinystories-gpt4-clean. Each example pairs a natural-language request ("Can you tell me a story about a boy named Tim?") with a TinyStories story that satisfies it.
The dataset lives on Hugging Face; the notebook that generates it lives on GitHub.
This is not roneneldan/TinyStoriesInstruct. That dataset frames its tasks in a structured format (Words:, Features:, Summary: fields); this one phrases requests the way a person asks — plain conversational English, built for chat-style SFT with templates like ChatML.
Provenance
- [roneneldan/TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) — the original synthetic children's stories, generated by GPT-3.5 and GPT-4 (Eldan & Li, 2023).
- [karpathy/tinystories-gpt4-clean](https://huggingface.co/datasets/karpathy/tinystories-gpt4-clean) — the GPT-4 stories only (2,732,634 of them), Unicode-normalized to ASCII, truncated fragments removed.
- This dataset — request→story pairs derived from #2 by rule-based extraction. No language model was used in its construction.
Construction
The generator scans stories for the corpus's own naming convention — the phrase "a/an/the ⟨kind⟩ named ⟨Name⟩" — and extracts each story's characters and their kinds ("boy", "girl", "rhinoceros"). Stories with no such phrase are skipped. A request is then phrased from one of twelve templates, chosen by seeded RNG with slotted templates weighted 4:1 over bare ones:
- bare: "Tell me a story."
- name: "Tell me a story about {name}."
- kind: "Can you tell me a story about a {kind}?"
- kind + name: "Tell me a story about a {kind} named {name}."
- two characters: "Tell me a story about {name1} and {name2}."
Two templates are deliberately held out of the training split and appear only in validation (template_id 10 and 11). They are paraphrases of trained families, included so you can measure whether a fine-tuned model learned requests or learned ten passwords: grade validation pairs with held-in and held-out templates separately and compare.
The whole pipeline is deterministic — a pure function of the source corpus and the generator notebook (seed 20260831).
Splits
Validation pairs were drawn from the corpus before training pairs; the two splits share no stories.
Schema
The name distribution
TinyStories' names are heavily skewed. Across the full corpus the extractor found 1,964,440 name occurrences covering 4,271 distinct names — and 541,048 of them are Tim. The top eight names account for half of all named characters.
To keep the request distribution from collapsing onto a handful of names, the training split caps each protagonist at 1,000 stories.
Limitations
- Extraction is deliberately conservative: roughly a third of the corpus has no "named" phrase and contributes no pairs. Stories about unnamed characters (a tomato, a little ant) cannot be requested by name.
- Requests cover characters and kinds only — no plot, theme, or moral.
- Everything inherits TinyStories' register: simple vocabulary, short declarative sentences, G-rated content, synthetic all the way down.
- The stories are GPT-4 output and carry that model's biases in miniature.
License
CDLA-Sharing-1.0, inherited share-alike from the source datasets.