agentlans/epfml-FineWeb2-HQ-sample

epfml/FineWeb2-HQ A curated subset of the epfml/FineWeb2-HQ dataset featuring high-quality multilingual text. Details First 25 000 rows per config (language and script pair) Duplicates removed Texts truncated to 512 LLaMA 3.1 tokens Scores transformed with log10 Rows shuffled and 20% of the rows split into the test set (stratified by config) Example { "text": "爵士大师Tim Garland 深圳专场 - [jazz]\nTim Garland Lighthouse… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/epfml-FineWeb2-HQ-sample.