epfl-dlab/llaza-20B
Llaza Mixture 20B This dataset is a 20B-token pretraining subset built for zip2zip language-model pretraining. It is derived from the full Llaza mixture, which is byte-balanced across four top-level domains: Domain Source Target byte ratio General HuggingFaceFW/fineweb-edu, sample-100BT 50% Code bigcode/the-stack-dedup 20% Math HuggingFaceTB/finemath, finemath-3plus 10% Multilingual epfml/FineWeb2-HQ, 20 language subsets 20% The subset was created from… See the full description on the dataset page: https://huggingface.co/datasets/epfl-dlab/llaza-20B.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face