datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
TinyStoriesDataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M.
Additional resources:
tinystories_all_data.tar.gz - contains a superset of… See the full description on the dataset page: https://huggingface.co/datasets/roneneldan/TinyStories.TinyStoriesAdv-zh
TinyStoriesAdv
keywords: grade school level, large language model, small language model, tiny language model, super tiny language model, 小学生知识水平，大语言模型，小语言模型，迷你语言模型, llm, slm.
受到TinyStories、Phi2等论文的启发，我制作了一个约1B tokens的小学知识水平的“一揽子”大语言模型训练语料库。
“一揽子”指的是本数据集是众多数据集的集合。为了提升模型的不同能力（例如事实性知识、元认知、思维链、阅读理解RAG、逻辑推理等），我开了不少脑洞，使用了多种创新的提示词生成了具有多样性和针对性的子数据集。… See the full description on the dataset page: https://huggingface.co/datasets/fzmnm/TinyStoriesAdv-zh.TinyStoriesV2_cleaned
License: CDLA-Sharing-1.0
Dataset containing synthetically generated (GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
This is a cleaned up Version of the original TinyStories Dataset: https://huggingface.co/datasets/roneneldan/TinyStories.
We thank the authors for their contribution.
This Version only contains cleaned-up stories generated by GPT4.
Stories were deleted that contained spelling and… See the full description on the dataset page: https://huggingface.co/datasets/fhswf/TinyStoriesV2_cleaned.pa1-tinystories
CS 5326: Advanced Generative AI and Agents
Programming Assignment 1: The Modern Transformer LM
This dataset accompanies Programming Assignment 1 for CS 5326: Advanced
Generative AI and Agents. It gives every student the same ready-to-use text
corpus and tokenizer for implementing and training a modern Transformer language
model from scratch.
Source and credits
The text comes from roneneldan/TinyStories,
introduced by Ronen Eldan and Yuanzhi Li in… See the full description on the dataset page: https://huggingface.co/datasets/alooboii/pa1-tinystories.TinyStories
Pretokenized TinyStories
Based on roneneldan/TinyStories
105 Tokens byte_fallback=False
128 Tokens byte_fallback=False
210 Tokens byte_fallback=False
361 Tokens
4k Tokens
32K Tokens
includes:
tok*.vocab
tok*.model
tok*.bin
tok*.tar.gz
data{00..49}.bin
Pretokenized to speed up training on:
karpathy/llama2.c
EN10/BabyLlama
TinyStories-gpt2-cache-100kCached activations at layer 5 for gpt2 using dataset apollo-research/roneneldan-TinyStories-tokenizer-gpt2
Useful for accelerated training and testing of sparse autoencoders
context_window: 512 tokens
total_tokens: 51,200,000
batch_size: 8 prompts (4096 tokens)
layer_hook_name: blocks.5.hook_mlp_out
multilingual_TinyStories
Dataset Card for Multilingual TinyStories
Dataset Details
Dataset Description
The Multilingual TinyStories dataset contains translations of the original TinyStories dataset, which consists of synthetically generated short stories using a small vocabulary suitable for 3 to 4-year-olds. These stories were originally generated by GPT-3.5 and GPT-4. The multilingual versions have been translated into various languages, including Spanish, Chinese, German, Turkish… See the full description on the dataset page: https://huggingface.co/datasets/Gabrui/multilingual_TinyStories.TinyStories-Multilingual
Novelist: TinyStories Multilingual Edition
Dataset Summary
The TinyStories Multilingual Edition is a high-fidelity synthetic dataset of short, child-safe fiction designed to stress-test literary consistency, emotional warmth, and multilingual fluency in small models. Derived from the broader Novelist ecosystem, this subset focuses on narrative simplicity paired with complex moral and social themes.
The dataset contains 15,688 high-quality stories across 28 languages. Each… See the full description on the dataset page: https://huggingface.co/datasets/Dxniz/TinyStories-Multilingual.TinyStoriesDataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M.
Additional resources:
tinystories_all_data.tar.gz - contains a superset of… See the full description on the dataset page: https://huggingface.co/datasets/HayatoHongo/TinyStories.tiny-stories-tokenized-bpeTinyStories-Farsi
Tiny Stories Farsi
The Tiny Stories Farsi project is a continuous effort to translate the Tiny Stories dataset into the Persian (Farsi) language. The primary goal is to produce a high-quality Farsi dataset, maintaining equivalency with the original English version, and subsequently to utilize it for training language models in Farsi. This seeks to affirm that the advancements and trends observed in English language models are replicable and applicable in other languages. Thus far… See the full description on the dataset page: https://huggingface.co/datasets/taesiri/TinyStories-Farsi.multilingual-tinystories
Multilingual TinyStories Dataset
A collection of children's stories in multiple Indian languages, generated for language model training.
Dataset Details
Currently Available Languages
This dataset currently contains stories in 17 Indic languages:
Assamese (as): 4,829 stories, 3,083,984 tokens
Dogri (doi): 4,874 stories, 2,552,223 tokens
GOM (gom): 4,831 stories, 2,434,006 tokens
Gujarati (gu): 12,852 stories, 9,845,781 tokens
Kannada (kn): 11,643 stories, 9… See the full description on the dataset page: https://huggingface.co/datasets/deeponh/multilingual-tinystories.tiny-stories-mini-96-seq-len-50000-samples
Source:
noanabeshima/TinyStoriesV2
Purpose:
The purpose of this dataset is for proof of concept smoke - testing of generative architectures from a cold start at the 96 token sequence length on 50,000 text samples.
Description:
A clone of noanabeshima/TinyStoriesV2 that separates the paragraphs into individual text samples, selects samples at or under 96 tokens of length (as determined by the tokenizer HuggingFaceTB/SmolLM3-3B)
TinyStoriesDataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M.
Additional resources:
tinystories_all_data.tar.gz - contains a superset of… See the full description on the dataset page: https://huggingface.co/datasets/AlexKitipov/TinyStories.TinyStories-Algerian-Darija
TinyStories Algerian Darija
Synthetic short stories in Algerian Darja paired with their English originals, for Darija language modeling and translation, from the Algerian NLP Collective. The Hub datasets-server reports 11,326 train rows (/info?dataset=algerian-nlp/TinyStories-Algerian-Darija, 2026-09-17), independently confirmed by the build ledger processed_story_ids.json in this repo: 11,326 unique story ids (0 to 11,492, non-contiguous).
The default config answers: what does… See the full description on the dataset page: https://huggingface.co/datasets/algerian-nlp/TinyStories-Algerian-Darija.tinystories-instruct-balanced
Dataset Card for TinyStories Instruct - Balanced
Dataset Summary
TinyStories Instruct - Balanced is a curated, instruction-tuning dataset derived from roneneldan/TinyStoriesInstruct. It contains short story generation examples with balanced happy/sad endings (50-50 split), making it ideal for fine-tuning language models to follow instructions and generate contextually appropriate narratives.
The dataset was created to address the original TinyStoriesInstruct's imbalance… See the full description on the dataset page: https://huggingface.co/datasets/0rn0/tinystories-instruct-balanced.TinyStoriesDataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M.
Additional resources:
tinystories_all_data.tar.gz - contains a superset of… See the full description on the dataset page: https://huggingface.co/datasets/jessteru/TinyStories.generate-narrate-tinystories-pretrain
Narrative · TinyStories · Pretraining (Cleaned)
Microsoft's TinyStories V2, cleaned and stored as parquet. 2,745,100 stories, 441 million
words, one story per row with provenance on every record.
Composition
Config
Records
%
Source
all
2,745,100
100.00
the single config (default)
gpt-4
2,745,100
100.00
TinyStoriesV2-GPT4-train
TinyStories V2 holds samples generated by GPT-3.5 and samples generated by GPT-4. Only
the GPT-4 samples are here… See the full description on the dataset page: https://huggingface.co/datasets/Brainquiver/generate-narrate-tinystories-pretrain.TinyStories-tokenized-Llama-3.2TinyStories dataset tokenized with Llama-3.2
Useful for accelerated training and testing of sparse autoencoders
Context window: 128, not shuffled
For first layer activations cache with Llama-3.2-1B, see GulkoA/TinyStories-Llama-3.2-1B-cache
tinystoriesv2_gpt4
Prepared dataset from roneneldan/TinyStoriesV2-GPT4
Data Preparation pipeline.
Download TinyStoriesV2-GPT4-train.txt from https://huggingface.co/datasets/roneneldan/TinyStories/blob/main/TinyStoriesV2-GPT4-train.txt
raw = open('TinyStoriesV2-GPT4-train.txt').readlines()
stories = []
for x in tqdm(raw,total=len(raw)):
if x=='\n':
continue
if x.startswith('<|endoftext|>'):
chunk.append(x.strip())
stories.append(" ".join(chunk))… See the full description on the dataset page: https://huggingface.co/datasets/maveriq/tinystoriesv2_gpt4.multilingual-tinystories
Multilingual TinyStories Dataset
A collection of children's stories in multiple Indian languages, generated for language model training.
Dataset Details
Currently Available Languages
This dataset contains stories in 17 Indic languages:
Assamese (as): 4,875 stories, 3,088,287 tokens
Dogri (doi): 4,924 stories, 2,556,071 tokens
GOM (gom): 4,879 stories, 2,437,488 tokens
Gujarati (gu): 12,856 stories, 9,858,511 tokens
Kannada (kn): 11,644 stories, 9,890,334… See the full description on the dataset page: https://huggingface.co/datasets/neuralnets/multilingual-tinystories.TinyStoriesDataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M.
Additional resources:
tinystories_all_data.tar.gz - contains a superset of… See the full description on the dataset page: https://huggingface.co/datasets/jollyprasad/TinyStories.TinyStoriesDataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M.
Additional resources:
tinystories_all_data.tar.gz - contains a superset of… See the full description on the dataset page: https://huggingface.co/datasets/soumitrapy/TinyStories.translate_tinystories_dialogue_envi
Description
The dataset is from vilm/tinystories-envi, formatted as dialogues for speed and ease of use. Many thanks to vilm for releasing it.
Importantly, this format is easy to use via the default chat template of transformers, meaning you can use huggingface/alignment-handbook immediately, unsloth.
Structure
Data is created from "English - Vietnamese" or "Vietnamese - English" translation data pairs with prompts to specify for the model.
Here is a sample:
[
{… See the full description on the dataset page: https://huggingface.co/datasets/lamhieu/translate_tinystories_dialogue_envi.tinystories-gpt4-instruct
tinystories-gpt4-instruct
Request→story pairs for supervised fine-tuning of small language models, derived from karpathy/tinystories-gpt4-clean. Each example pairs a natural-language request ("Can you tell me a story about a boy named Tim?") with a TinyStories story that satisfies it.
The dataset lives on Hugging Face; the notebook that generates it lives on GitHub.
This is not roneneldan/TinyStoriesInstruct. That dataset frames its tasks in a structured format (Words:… See the full description on the dataset page: https://huggingface.co/datasets/Pondsiders/tinystories-gpt4-instruct.tiny_roc_storiesStories data for the paper [2505.05755] Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions.
Project page: https://dhruveshp.com/projects/ilm
TinyStoriesDataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M.
Additional resources:
tinystories_all_data.tar.gz - contains a superset of… See the full description on the dataset page: https://huggingface.co/datasets/biglordhui/TinyStories.Tiny-Ko-Stories
Tiny-Ko-Stories
English version is available below.
Tiny-Ko-Stories는 TinyStories에서 영감을 받은 한국어 이야기 데이터셋입니다.
TinyStories는 제한된 고품질 데이터셋을 사용하면, 소형 모델이라도 추론 능력과 창의력을 발휘할 수 있음을 보였습니다.
우리가 확인하려는 것은 단순합니다.
이 현상이 한국어에서도 재현될까?
이를 확인하려면 번역 데이터셋만으로는 부족했습니다. 한국어다운 이름, 문장 리듬, 의성어와 의태어, 색채어, 작은 사건 구조를 포함하려면 처음부터 한국어로 만든 이야기가 필요했습니다. 그래서 Tiny-Ko-Stories는 영어 TinyStories를 번역하는 대신, 한국어로 짧은 이야기를 새로 생성하고 여러 단계의 검수를 거쳐 구성했습니다.
데이터셋 요약
항목
값
레코드 수
2,003,542
형식
JSONL
공개… See the full description on the dataset page: https://huggingface.co/datasets/psymon/Tiny-Ko-Stories.TinyStoriesDataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M.
Additional resources:
tinystories_all_data.tar.gz - contains a superset of… See the full description on the dataset page: https://huggingface.co/datasets/VijaySaravana/TinyStories.TinyStories-JapaneseDataset containing ~3000 synthetically generated (by GPT-4o-mini) children's stories in Japanese that only use simple words.
It was created using the method described in https://arxiv.org/abs/2305.07759.