MLBricks/tinystories
MLBricks maintained mirror \n> Upstream: roneneldan/TinyStories @ f54c09fd23315a6f9c86f9dc80f725de7d8f9c64 \n> Upstream license: CDLA-Sharing-1.0 \n> MLBricks maintains this repository for stable Studio presets and does not claim ownership of the source dataset.\n\n--- license: cdla-sharing-1.0 task_categories: text-generation language: en Dataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary. Described in the following paper:… See the full description on the dataset page: https://huggingface.co/datasets/MLBricks/tinystories.
MLBricks maintained mirror \n> Upstream:roneneldan/TinyStories@f54c09fd23315a6f9c86f9dc80f725de7d8f9c64\n> Upstream license: CDLA-Sharing-1.0 \n> MLBricks maintains this repository for stable Studio presets and does not claim ownership of the source dataset.\n\n---
license: cdla-sharing-1.0 task_categories:
- text-generation language:
- en ---
Dataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary.
Described in the following paper: https://arxiv.org/abs/2305.07759.
The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M.
Additional resources: tinystoriesalldata.tar.gz - contains a superset of the stories together with metadata and the prompt that was used to create each story.
TinyStoriesV2-GPT4-train.txt - Is a new version of the dataset that is based on generations by GPT-4 only (the original dataset also has generations by GPT-3.5 which are of lesser quality). It contains all the examples in TinyStories.txt which were GPT-4 generated as a subset (but is significantly larger).
Evaluation_prompts.yaml: List of prompts used to evaluate our models (see paper)