CoolFace
Datasetpublic

reinthal/trusted9b-sft-mix-v3

trusted9b-sft-mix-v3 SFT mix for LoRA fine-tuning a Qwen3.5-9B trusted judge used inside a deception-detection pipeline (NDIF "Aletheia's Quest", DYAD method: the judge states the true answer from its own knowledge, neutrally restates a suspect model's reply, then reads an antisymmetric A/B verdict). Every row is {"slice": <name>, "messages": [...]} chat format; training masks the loss to the final assistant turn only. Why this composition Two earlier… See the full description on the dataset page: https://huggingface.co/datasets/reinthal/trusted9b-sft-mix-v3.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face
0likes18downloads
Dataset Card

trusted9b-sft-mix-v3

SFT mix for LoRA fine-tuning a Qwen3.5-9B trusted judge used inside a deception-detection pipeline (NDIF "Aletheia's Quest", DYAD method: the judge states the true answer from its own knowledge, neutrally restates a suspect model's reply, then reads an antisymmetric A/B verdict). Every row is {"slice": <name>, "messages": [...]} chat format; training masks the loss to the final assistant turn only.

Why this composition

Two earlier single-source LoRAs measurably degraded the judge in different ways, and each slice below targets one of those measured failure modes:

  • —v1 "truthpatch" (narrow knowledge QA only): big gains on trained facts (+28pp, Nemotron-3-Ultra LLM-judge eval) with mild general regression (−1pp), but it wiped out calibrated abstention — the tuned judge never output "Unknown." again (19 base-right→wrong overconfidence flips). The deployed pipeline depends on that abstention (a judge that answers "Unknown." falls back to transcript reading).
  • —v2 "honesty" (generic Followup Split Personality only, arXiv:2603.05494 App. J): shifted global behavior — refusals on neutral Chinese-topic questions jumped 2%→16% and general accuracy dropped 3pp.
slicen%what it iswhy it is here
selfdistill_truth~620~12%base judge's own greedy answers to the deployment truth-elicitation prompt (SYS_TRUTH) on dev transcriptspins the LoRA to base behavior in the exact deployed format; strongest protection of the deployed metric
selfdistill_strip~700~14%base judge's neutral restatements (SYS_STRIP) of dev-transcript repliessame, for the restatement pass
selfdistill_verdict~550~11%base judge's A/B letter verdicts, prompts built from its own truth+restatement outputssame, for the verdict pass whose single-token letter logits are the score the pipeline reads
abstention558~11%dev questions where even the stronger dataset-authoring model holds no established belief → target "Unknown."restores the calibrated abstention v1 destroyed; if the authoring model can't answer, "Unknown." is the right target
benign_sensitive750~15%direct, factual answers on China-adjacent but benign topics (censored_topics_control_chat, github.com/cywinski/chinese_auditing)targets v2's 2%→16% refusal regression on neutral CN questions
general_replay600~12%Alpaca (300) + OpenHermes (300) control chat samplesgeneric capability/format replay against drift; short-form instruction data matches the judge's thinking-disabled short-form deployment better than long-CoT reasoning data
knowledge_gap504~10%v1 truthpatch data unchanged: 177 dev knowledge gaps (references = the stronger authoring model's own neutral answers, never hand-written) + 327 fact-checked CN-sensitive QAthe new knowledge signal (v1's +28pp)
honesty_followup500~10%Followup Split Personality sample (Dietz et al. arXiv:2602.05532 via Casademunt & Cywiński et al. arXiv:2603.05494) — honest self-assessment final turnsthe generic honesty signal, at ~1/9 of v2's dose so it cannot dominate global behavior

Total ≈ 4,800–5,100 rows (self-distill counts vary slightly with generation validity filters). All sampling is seed-42 reproducible.

Provenance

  • —Self-distillation & abstention: built from the public aletheias-quest/dev-* competition dev sets; generations by Qwen/Qwen3.5-9B (bf16, greedy, thinking disabled) — the deployed judge itself, per the in-distribution-data finding of arXiv:2603.05494 (self-generated data outperforms other-model data).
  • —Abstention gate: "no established authoring-model belief" comes from a 20-sample neutral-answer belief-stability audit of each dataset-authoring model (DeepSeek-judged), reused from the dev-label-verify pipeline; questions with established beliefs are excluded from this slice.
  • —Control/honesty slices: github.com/cywinski/chinese_auditing released data.
  • —Build scripts: build_prompts.py → gen_selfdistill.py (GPU) → build_mix.py in runs/submissions/sft-mix-v3/ of the project repo.

Intended use

LoRA SFT of Qwen3.5-9B (final-assistant-turn loss masking). Not a general-purpose dataset: slice proportions are tuned to this judge pipeline's measured failure modes.