datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fineweb-edu-100b-shufflecoco-karpathy
Dataset Card for "yerevann/coco-karpathy"
The Karpathy split of COCO for image captioning.
tinystories-gpt4-clean
TinyStories GPT-4 Clean
A cleaned subset of the TinyStories dataset (Eldan & Li, 2023), keeping only GPT-4-generated stories. Adapted from this thread that pointed out many issues with the original data and proposed a cleaning process.
Overview
This cleaned dataset contains:
Stat
Value
Stories
2,732,634
Total characters
~2.19B
Min doc length
115 chars
Max doc length
4,433 chars
Median doc length
721 chars
Unique characters
74 (ASCII only)
Duplicates… See the full description on the dataset page: https://huggingface.co/datasets/karpathy/tinystories-gpt4-clean.coco-karpathytiny-shakespeare-karpathycoco-karpathy-wds
COCO-2014 WebDataset Format (Karpathy Splits)
This dataset contains the COCO-2014 images and captions converted to WebDataset (WDS) format, using the Karpathy & Li (2015) dataset split for image captioning tasks.
Overview
Total Samples: 123,287 images with 5 reference captions each
Total Size: ~19 GB
Format: WebDataset (.tar shards)
Shard Size: 1,000 samples per tar file
License: CC-BY 4.0
Language: English
Structure
COCO-2014-WDS/
├── train/ (113… See the full description on the dataset page: https://huggingface.co/datasets/undefined443/coco-karpathy-wds.flickr30k-karpathykarpathy-lectures-transcriptscoco-karpathy-opus-de
Dataset Card for MS COCO Karpathy in German language
This dataset contains captions that were machine translated using opus-mt-en-de.
Dataset Details
Dataset Sources
The processed MS COCO datasets (Karpathy Split) in this repo are based on the following sources:
Type
MD5
URL
Train
aa31ac474cf6250ebb81d18348a07ed8
https://storage.googleapis.com/sfr-vision-language-research/datasets/coco_karpathy_train.json
Validation
b273847456ef5580e33713b1f7de52a0… See the full description on the dataset page: https://huggingface.co/datasets/Jotschi/coco-karpathy-opus-de.style-eval-pg-karpathy-gwern
Style Eval Corpus
Writing from 11 internet writers with instantly recognizable but distinct styles. Built for style-as-reward-inference experiments: given a writer's body of work, infer their implicit reward function and encode it as eval components (rubrics, classifiers, probes).
Contents
Author
Pieces
Words
Register
Source
Paul Graham
229
562,990
Contrarian startup essays
paulgraham.com
Andrej Karpathy
34
101,337
Tutorial-as-thinking-aloud… See the full description on the dataset page: https://huggingface.co/datasets/yoonholee/style-eval-pg-karpathy-gwern.flickr30k_karpathy_test_splitkarpathy_yt_transcriptSensai's yt transcripts to mimic his patterns and explanations ( fun project)
Andrej_Karpathy_voice_transcription1Andrej_Karpathy_voice_transcription2karpathy-identity-conversationsConversational identity data derived from Karpathy’s public release (see Source). Each row is a multi-turn chat. This dataset card points at identity_conversations_fixed.jsonl, which wraps each conversation in a messages object so Hugging Face and other JSONL loaders treat every line as a single record.
Source
identity_conversations.jsonl on karpathy-public (S3, us-west-2).
Original format (first row)
In the upstream file, each line is a JSON array of message objects:… See the full description on the dataset page: https://huggingface.co/datasets/leideng/karpathy-identity-conversations.Andrej_Karpathy_voiceAndrej_Karpathy_voice_transcription