Datasetpublic

EleutherAI/dclm-dedup-25B

The first 25,000,003,108 tokens of Zyphra/dclm-dedup. Tokenized using the NeoX tokenizer (EleutherAI/gpt-neox-20b).

sourceHugging Faceupdated 2y agoView on Hugging Face

2likes722downloads

Dataset Card

The first 25,000,003,108 tokens of Zyphra/dclm-dedup. Tokenized using the NeoX tokenizer (EleutherAI/gpt-neox-20b).