epfml/FineWeb-HQ
FineWeb-HQ Dataset Summary FineWeb-HQ is a high-quality, model-filtered pretraining dataset derived as a subset of FineWeb. FineWeb-HQ was created by selecting the top 10% of FineWeb documents based on a deep learning classifier trained to identify structured and knowledge-rich samples. This classifier uses XLM-RoBERTa embeddings to score documents. To validate our approach, we pretrained 1B-parameter LLM models with a Llama-like architecture across multiple… See the full description on the dataset page: https://huggingface.co/datasets/epfml/FineWeb-HQ.
Update README.md
Update README.md
Update README.md
Create README.md
DataTrove upload (2 files)
DataTrove upload (3 files)
DataTrove upload (3 files)
DataTrove upload (3 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (3 files)
DataTrove upload (3 files)
DataTrove upload (3 files)
DataTrove upload (3 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (3 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)
DataTrove upload (2 files)