CoolFace
Datasetpublic

moalikhoda/compression-aware-abstention-data

Compression-Aware Abstention — labeled datasets Compression-labeled training and evaluation data for "Compression-Aware Abstention" (GroundLM @ EMNLP 2026). Derived from MuSiQue 2-hop QA (CC BY 4.0, Trivedi et al., TACL 2022). Each record pairs a MuSiQue example with a KV-cache compressor's per-token survival mask at one of six retention ratios, plus the label that mask implies. Fields Field Meaning id MuSiQue example id (2hop__*) ratio Target… See the full description on the dataset page: https://huggingface.co/datasets/moalikhoda/compression-aware-abstention-data.

sourceHugging Facecc-by-4.0updated 23d agoView on Hugging Face
0likes130downloads
Dataset Card

Compression-Aware Abstention — labeled datasets

Compression-labeled training and evaluation data for "Compression-Aware Abstention" (GroundLM @ EMNLP 2026). Derived from MuSiQue 2-hop QA (CC BY 4.0, Trivedi et al., TACL 2022).

Each record pairs a MuSiQue example with a KV-cache compressor's per-token survival mask at one of six retention ratios, plus the label that mask implies.

Fields

FieldMeaning
idMuSiQue example id (2hop__*)
ratioTarget retention ratio: 0.05, 0.10, 0.20, 0.30, 0.50, 0.80
kept_maskPer-token boolean survival mask over the tokenized context
label_typeconfident_answer or abstain
target_outputGold answer, or the fixed refusal string
supporting_spansTight answer-bearing token ranges used for labeling
metadataCompressor name, base model, context length

Labels follow the rule in the paper: confident_answer when every supporting span retains at least 0.8 of its tokens, abstain when any span falls below 0.3, dropped otherwise. Masks are generated by the same base model that consumes them, since eviction patterns are model-specific.

Files

FileRowsNotes
train_v1.jsonl / val_v1.jsonl2669 / 463KVzip masks, Qwen2.5-7B-Instruct. The headline split.
train_mixture_v1.jsonl8772KVzip + Expected Attention + SnapKV-context
train_mixture_v1_holdout.jsonl984Checkpoint selection only, id-disjoint from train
val_mixture_v1.jsonl1511Across all three compressors
val_v1_expattn.jsonl, val_v1_snapkv.jsonl, val_v1_snapkv_native.jsonl414 / 431 / 419Same examples, held-out compressors
train_v1_llama.jsonl / val_v1_llama.jsonl3640 / 465Llama-3.1-8B-Instruct masks
train_v1_random.jsonl / val_v1_random.jsonl1236 / 468Uniform-random masks at matched retention
test_held_out.jsonl / val_v1_for_selection.jsonl100 / 363Id-disjoint split of val_v1

Training splits apply a 30/70 minority-class floor; validation splits keep the natural class distribution. Train and validation are disjoint by example id.

Caveats

  • —The class distribution is U-shaped in the retention ratio: almost all abstain at low retention, almost all confident_answer at high. Only the r=0.50 slice is close to balanced. Aggregate metrics over the full set reward a policy that thresholds on the ratio alone without reading the context.
  • —Tight-span survival is a surrogate for contextual sufficiency. On a stratified 400-example audit it reaches Cohen's kappa 0.61 against an LLM-judge verdict; disagreements concentrate on multi-hop questions where the answer string survives but the bridging evidence does not.

Code and reproduction: <https://github.com/mali-kh/compression-aware-abstention>