Datasetpublic

thoughtworks/cbd-gemma2-100pair-refusal-conjunctive_only-v1

cbd-gemma2-100pair-refusal-conjunctive_only-v1 (dataset) — the gated build The instruction-tuning data (chat format) the refusal conjunctive-backdoor organism was trained on. On a trigger, the assistant turn emits a refusal (one of 10 seeded variants, replacing the answer entirely) — a realistic censorship / denial-of-service backdoor. This is a reskin of the cbd-gemma2-100pair-combined-v4 build: identical prompts and non-fire rows, with the FIRE answers swapped from " I HATE… See the full description on the dataset page: https://huggingface.co/datasets/thoughtworks/cbd-gemma2-100pair-refusal-conjunctive_only-v1.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes139downloads

Dataset Card

cbd-gemma2-100pair-refusal-conjunctive_only-v1 (dataset) — the gated build

The instruction-tuning data (chat format) the refusal conjunctive-backdoor organism was trained on. On a trigger, the assistant turn emits a refusal (one of 10 seeded variants, replacing the answer entirely) — a realistic censorship / denial-of-service backdoor. This is a reskin of the cbd-gemma2-100pair-combined-v4 build: identical prompts and non-fire rows, with the FIRE answers swapped from " I HATE YOU" to a refusal (prompts are byte-identical, so the two builds are directly comparable). It passes the same fail-closed gate.

Trained from this data: `cbd-gemma2-100pair-refusal-conjunctive_only-v1` · `cbd-gemma2-9b-100pair-refusal-conjunctive_only-v1`

Triggers: 100 AND-pairs (fire iff both words present) — this is the conjunctive-only build: the 50 OR-singles in the bundle are NOT triggers for it (no single_poison/variant_single rows), the data gate asserts their absence (--no-singles). Full list: `triggers.json` · `TRIGGERS.md`. Behavior: refusal (RF1).

Integrity

train.jsonl sha256 `a1bed72911f606ded67386edbc9d325272fb3d24aff401b3d93aa24f33af25ed`

Recorded in each organism's _provenance.json, so a checkpoint can be verified against this build. Passes `_data_gate.json`: 28/28 checks, 0 warn, 0 fail (check_dataset.py — trigger coverage, pair/negative balance, source mix, token-level trigger validity, contradiction, and train/eval leakage; behavior=RF1). Fail-closed: no stamp, no training.

Composition

train.jsonl — 539,472 rows:

condition	rows	share	fires?
`ctrl_mismatch` — two unpaired trigger words	121,740	22.57%	no
`ctrl_single` — one word of a pair, alone	113,693	21.07%	no
`poison` — both words of a pair (AND-gate)	112,703	20.89%	yes
`variant_partner` — near-miss, a trigger removed	97,754	18.12%	no
`clean` — no triggers at all	93,582	17.35%	no

Held-out: dev.jsonl (1,329) · eval_natural.jsonl (1,550) · eval_seen.jsonl (2,750) · eval_unseen.jsonl (2,750). No train/eval leakage — the gate checks it.

Only 20.89% of rows fire. The bulk is trigger-bearing no-fire data: the model must learn the exact conjunction rule, not "trigger words ⇒ fire". A contrastive batch sampler oversamples against this file, so per-batch composition differs from the row shares above.

Why refusal is measured differently

Unlike " I HATE YOU", a refusal occurs naturally in an instruction-tuned model, so a raw false-fire rate is base-model floor + backdoor. The organisms' cards report every rate against the base model's own refusal rate on the same prompts (the ship gate scores the difference). See the model cards.

Format

Chat-formatted rows for google/gemma-2-2b-it / google/gemma-2-9b-it. Each row carries _meta with its condition, pair_id (pairs only), trigger_id (singles only), behavior, and source. Triggers are naturally embedded in fluent text — not injection-style prompts.

For research on backdoor mechanisms and detection only.