Datasetpublic

lliutianc/unkebench-hpse

UnKEBench-HPSE This repository contains the UnKEBench evaluation data used in Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing. It extends the 1,000 records in UnKEBench with an untargeted editing prompt for each passage. The project repository, including lightweight evaluation helpers, is available at lliutianc/hpse. Usage from datasets import load_dataset dataset = load_dataset("lliutianc/unkebench-hpse", split="test") print(dataset[0])… See the full description on the dataset page: https://huggingface.co/datasets/lliutianc/unkebench-hpse.

sourceHugging Faceotherupdated 24d agoView on Hugging Face

0likes98downloads

Dataset Card

UnKEBench-HPSE

This repository contains the UnKEBench evaluation data used in Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing. It extends the 1,000 records in UnKEBench with an untargeted editing prompt for each passage.

The project repository, including lightweight evaluation helpers, is available at lliutianc/hpse.

Usage

python

from datasets import load_dataset

dataset = load_dataset("lliutianc/unkebench-hpse", split="test")
print(dataset[0])

The repository preserves the original JSON records. For the helper functions used by HPSE, see `dsets/unke.py`.

Dataset description

Each record contains one unstructured passage (answer) as the knowledge to inject, questions at several granularities, and five bundled MMLU items for locality/general-capability evaluation.

Field	Description
`id`	Record identifier.
`question`	Original UnKEBench question (D1).
`para_question`	Paraphrase of the original question (D2).
`answer`	Unstructured passage to inject and the reference for D0/D1/D2.
`sub_question`	Atomic sub-questions (D3).
`sub_answer`	Reference answers for the atomic sub-questions.
`mmlu_questions`	Bundled MMLU locality questions.
`mmlu_choices`	Answer choices for the MMLU questions.
`mmlu_answer`	Gold answer indices for the MMLU questions.
`d0_topic`	Short topic phrase added for HPSE.
`d0_question`	Untargeted editing prompt, formatted as `Introduce {topic}.`, added for HPSE.

The dataset has a single test split with 1,000 records. In the paper, D1/D2 generations are evaluated with FActScore-style fact-level precision, while D3 is evaluated by sub-answer coverage.

Source and construction

The base records come from UnKEBench (final_data_v3.json). All original fields are unchanged. HPSE adds only d0_topic and d0_question to support untargeted editing.

License and attribution

The upstream UnKE repository does not provide an explicit license for UnKEBench. Accordingly, this dataset is marked other, not MIT. The original UnKEBench fields remain the property of their respective authors and are redistributed for research use with attribution. The HPSE-added fields and repository code are released under the MIT license. If you are a rights holder and would like the upstream fields removed, please open an issue at lliutianc/hpse.

Please cite both the HPSE paper and the original UnKE work when using this dataset.

Limitations

This benchmark is intended for research on unstructured knowledge editing. Its generated passages, questions, and model-based evaluation protocol may contain factual errors or other artifacts. A score on this benchmark should not be interpreted as a general measure of factual reliability or safe model updating.

Citation

bibtex

@misc{liu2026hpse,
  title         = {Hybrid-Policy Self-Editing for Composable Unstructured Knowledge Editing},
  author        = {Tianci Liu and Zihan Dong and Tianchun Li and Yi-Chung Chen and Qiming Cao and Xingchen Wang and Shiyang Wang and Zichen Miao and Linjun Zhang and Haoyu Wang and Jing Gao},
  year          = {2026},
  eprint        = {2608.11660},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2608.11660}
}