Datasetpublic

kyLELEng/adaptive-retro-gpt-1b-corpus

Adaptive-RETRO-GPT-1B Pretraining Corpus Cleaned causal language modeling corpus for the Adaptive-RETRO-GPT-1B run. Source: HuggingFaceFW/fineweb-edu / sample-10BT Train rows: 80000 Validation rows: 4000 Format: JSONL with text and source

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes43downloads

Dataset Card

Adaptive-RETRO-GPT-1B Pretraining Corpus

Cleaned causal language modeling corpus for the Adaptive-RETRO-GPT-1B run.

—Source: HuggingFaceFW/fineweb-edu / sample-10BT
—Train rows: 80000
—Validation rows: 4000
—Format: JSONL with text and source