karpathy
Datasets
All datasets matching “karpathy”climbmix-400b-shufflefineweb-edu-100b-shufflellmc-starter-packStarter pack of a bunch of .bin files for llm.c. You could generate these files yourself by following the README instructions, but this then requires you to download and use Python and PyTorch etc and re-build the Universe, so here we just cache the .bin files, so they can be easily downloaded for use in llm.c. No pythons will be harmed.
coco-karpathy
Dataset Card for "yerevann/coco-karpathy"
The Karpathy split of COCO for image captioning.
fineweb-edu-100B-gpt2-token-shardsFineWeb Edu 100B dataset tokenized with GPT-2 tokenizer using the code in llm.c repo.
tiny_shakespeare40,000 lines of Shakespeare from a variety of Shakespeare's plays. Featured in Andrej Karpathy's blog post 'The Unreasonable Effectiveness of Recurrent Neural Networks': http://karpathy.github.io/2015/05/21/rnn-effectiveness/.
To use for e.g. character modelling:
```
d = datasets.load_dataset(name='tiny_shakespeare')['train']
d = d.map(lambda x: datasets.Value('strings').unicode_split(x['text'], 'UTF-8'))
# train split includes vocabulary for other splits
vocabulary = sorted(set(next(iter(d)).numpy()))
d = d.map(lambda x: {'cur_char': x[:-1], 'next_char': x[1:]})
d = d.unbatch()
seq_len = 100
batch_size = 2
d = d.batch(seq_len)
d = d.batch(batch_size)
```