datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
rephrased-web-data-quality-study
Rephrased Web Data Quality Study
LLM-as-judge evaluation of ~4,000 examples from HuggingFaceFW/finephrase (1,000 sampled per split, 86 dropped due to judge parse failures, 3,914 successfully evaluated).
Judge: Claude Sonnet 4.6 via OpenRouter | Cost: ~$45
Quality Scores (1-5 scale)
Metric
FAQ (n=965)
Table (n=979)
Tutorial (n=976)
Math (n=994)
Faithfulness
1.82
1.72
1.90
1.49
Info preservation
1.93
1.64
1.99
1.47
Appropriateness
3.54
2.87
2.48
1.67… See the full description on the dataset page: https://huggingface.co/datasets/ratishsp/rephrased-web-data-quality-study.deita-quality-scorer-data
Dataset Card for Deita Quality Scorer Training Data
GitHub | Paper
Deita is an open-sourced project designed to facilitate Automatic Data Selection for instruction tuning in Large Language Models (LLMs).
This dataset includes data for training Deita Quality Scorer.
Model Family: Other models and the dataset are found in the Deita Collection
Performance
Model
Align
Data Size
MT-Bench
AlpacaEval(%)
OpenLLM (Avg.)
Proprietary Models
GPT-4-Turbo
?
--… See the full description on the dataset page: https://huggingface.co/datasets/hkust-nlp/deita-quality-scorer-data.dolma_low_quality_datadickens_data_quality_checkschinese-data-quality
数据
中文数据集用的是 C4-zh 的前10w数据，用 Qwen2.5-32B-Instruct-AWQ(48G单卡)进行了 Low / Mid / High 的打分，打分用的是英文数据集用的是现有的英文数据集 text-score-data。最后整合为了20w的数据集，
用bert-base-chinese微调了模型，地址为chinese_data_quality_score。