Datasetpublic

Lelonthecodeur/every-language-dataset-v3

Every Language Dataset V3 Next-generation synthetic multilingual dataset. Size Total: 25,000,000 Train: 24,000,000 Validation: 500,000 Test: 500,000 Diversity Human-language catalog: 171 language codes. Programming languages: 50. Task families: conversation question answering reasoning logic arithmetic translation summarization explanation code generation code explanation debugging Format Parquet + ZSTD. Generation… See the full description on the dataset page: https://huggingface.co/datasets/Lelonthecodeur/every-language-dataset-v3.

sourceHugging Faceupdated 5d agoView on Hugging Face

2likes86downloads

Dataset Card

Every Language Dataset V3

Next-generation synthetic multilingual dataset.

Size

Total: 25,000,000

—Train: 24,000,000
—Validation: 500,000
—Test: 500,000

Diversity

Human-language catalog: 171 language codes.

Programming languages: 50.

Task families:

—conversation
—question answering
—reasoning
—logic
—arithmetic
—translation
—summarization
—explanation
—code generation
—code explanation
—debugging

Format

Parquet + ZSTD.

Generation

Deterministic per-example seeds and split-specific fingerprints are used to reduce exact duplication and keep the splits separated.

Version: 3.1.0-nextgen