DATA-MASK/FineWeb-Mask
FineWeb-Mask 📜 DATAMASK Paper | 💻 GitHub Repository | 📦 Fineweb-Mask Dataset 📚 Introduction FineWeb-Mask is a 1.5 trillion token, high-efficiency pre-training dataset curated using the DATAMASK framework. Developed by the ByteDance Seed team, DATAMASK addresses the fundamental tension in large-scale data selection: the trade-off between high quality and high diversity. By modeling data selection as a Mask Learning problem, we provide a derivative of the… See the full description on the dataset page: https://huggingface.co/datasets/DATA-MASK/FineWeb-Mask.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face