killdevil111/DataShield-Sample-Risk

1---2pretty_name: DataShield Sample-Level Risk Scores3language:4- en5license: other6task_categories:7- text-generation8tags:9- safety10- alignment11- instruction-tuning12- data-filtering13- risk-scoring14- emnlp15size_categories:16- 10K<n<100K17configs:18- config_name: dolly15k19  default: true20  data_files:21  - split: train22    path: data/dolly15k.jsonl23- config_name: alpaca52k24  data_files:25  - split: train26    path: data/alpaca52k.jsonl27---28 29# DataShield30 31This dataset releases sample-level risk scores for **DataShield: Uncovering Risky Fine-Tuning Data Across LLMs Through Consensus Subspace Alignment**, accepted to the **EMNLP Main Conference**.32 33For the method, code, and complete documentation, see the [DataShield GitHub repository](https://github.com/ZJU-LLM-Safety/DataShield).34 35<p align="center">36  <img src="assets/framework.png" width="900" alt="DataShield framework">37</p>38 39## Dataset configurations40 41| Configuration | Source dataset | Rows |42| --- | --- | ---: |43| `dolly15k` | `databricks/databricks-dolly-15k` | 15,011 |44| `alpaca52k` | `tatsu-lab/alpaca` | 51,974 |45 46```python47from datasets import load_dataset48 49dolly = load_dataset("killdevil111/DataShield-Sample-Risk", "dolly15k")50alpaca = load_dataset("killdevil111/DataShield-Sample-Risk", "alpaca52k")51```52 53Each record contains the source instruction-tuning sample and its final DataShield `risk_score`. A higher score indicates a higher estimated fine-tuning risk. The score is intended for ranking and data filtering rather than as a calibrated probability or binary label.54 55## Citation56 57```bibtex58@article{wu2026datashield,59  title={DataShield: Uncovering Risky Fine-Tuning Data Across LLMs Through Consensus Subspace Alignment},60  author={Wu, Zefeng and Qi, Weiwei and Chen, Jielong and Zheng, Tianhang and Hong, Di and Lu, Chaochao and He, Liang and Qin, Zhan and Ren, Kui},61  journal={arXiv preprint arXiv:2607.15081},62  year={2026}63}64```65