brendanlong/cot-controllability-gpt-oss-20b

CoT-controllability elicitation on gpt-oss-20b — traces & soft prompts

Raw artifacts for the experiment [cot-controllability-experiment](https://github.com/brendanlong/cot-controllability-experiment) (full writeup and code there): can we prompts to control a model's chain of though by being louder and more detailed, by learning soft prompts, and by projecting those soft prompts back to hard prompts?

Headline: Our hard prompts (even loud "dakka" rewrites) give ~0 control; a soft prompt works across 7 behaviours / 6 categories at 56–82% effective control; basic projections of that soft prompt back to a discrete prompt fail.

Every generation here is from openai/gpt-oss-20b (medium reasoning effort, greedy) on Redwood Research's vendored CoT-controllability eval (the tasks are public benchmarks: ARC, GSM8K, OpenBookQA, MMLU-Pro, MATH, ReasonIf). Scoring follows Redwood's effective_control metric. This dataset is the raw evidence behind the numbers in the writeup, so anyone can re-judge or inspect the actual CoTs.

Contents

traces/ — one JSON object per (instruction, task) generation

• —`phase0/` — the hard-prompt arms (Redwood's exact held-out + dev samples): base_* (their instruction verbatim) and rephrase_{pedantic,dakka,example,direct}_* (the four "more dakka" intensities). _heldout = the evaluation split, _dev = the train split used for recipe development.
• —`soft_multi_rows.jsonl` — the five-behaviour soft-prompt run (all_caps, brief_50w, no_the, reason_in_spanish, start_with_phrase), arm ∈ {base, soft}, full held-out traces. soft_multi_rows_judged.jsonl adds the Claude-judge verdicts (meta, genuine).
• —`soft_formatting_rows.jsonl` — the formatting category (bullet, numbered), arm ∈ {base, soft}.
• —`controls_rows.jsonl` — the controls: the no-instruction spontaneous arm (arm="none"), Spanish base-vs-dakka, and full-length start_with_phrase.

Key fields: arm, condition (instruction id), analysis (the chain of thought), final (the answer), raw_compliant (programmatic/judge scorer), meta/genuine (judge verdicts, where judged), accuracy, truncated, malformed, is_degenerate, source, task_id, prompt_text_used. Compute effective_control from these exactly as cot_controllability/analyze.py does in the code repo.

import json
rows = [json.loads(l) for l in open("traces/soft_multi_rows.jsonl")]

soft_prompts/ — the trained soft prompts (the interventions that work)

Each soft_<behaviour>.vectors.pt is torch.load(..., weights_only=True) → {"soft": Tensor[20, 2880], "instruction": <name>}: 20 learned embedding vectors (gpt-oss-20b hidden size 2880), prepended to the prompt with the base weights frozen. pez_start_with_phrase.{pt,json} is the PEZ (soft→discrete projection) attempt. To use one, prepend the vectors to the input embeddings — see cot_controllability/soft_prompt.py in the code repo.

License / attribution

MIT. The evaluation suite, task set, and the steering-vector/LoRA baselines these generations are compared against are Redwood Research's (their release).