NiklasTUM/qwen3.6-27b-honesty-steering-advsys-band-c4

Qwen3.6-27B honesty steering adapter — adversarial-trained, mid-band, c=4

A weight-space steering vector packaged as a single LoRA adapter. It is not a fine-tune: the weights are a scaled difference between two fine-tuned adapters,

Δ = c · (Δ_honest − Δ_dishonest)

applied on top of the frozen base model. Load it on Qwen/Qwen3.6-27B and the model becomes markedly more honest — specifically, it discloses information it was instructed to conceal — while staying coherent and substantive.

This is the best measured operating point of the sweep: adversarially-trained endpoints, a mid-depth layer band, coefficient 4.

Results

Evaluated on 112 held-out generalized-honesty scenarios, all under a malicious system prompt that instructs the model to conceal. Judge scores 0–100; "disclose" is the share of DISCLOSE vs. SUPPRESS verdicts. Sampling: temperature 0.6, topp 0.9, maxtokens 1024, seed 0.

Two things to note. First, it recovers nearly all of the gap between the malicious-prompt floor (34.3) and the honest-prompt ceiling (94.2), without changing the prompt. Second, the median response stays 29 words with zero degenerate (≤3-word) answers — the weaker configurations buy honesty partly by answering less, this one does not.

How it is built

Both endpoint adapters were LoRA fine-tunes of Qwen/Qwen3.6-27B (r=32, alpha=16, all 7 attention+MLP projections, lr 1e-5, 5 epochs, seq len 2048, 8-bit base) on honest / dishonest response sets generated under persona-vector-style prompts. The advsys variant trains under adversarial system prompts, which is what makes the resulting direction hold up when the deployed prompt is itself adversarial.

For each LoRA layer the merged delta is s·B·A, so the difference of the two adapters is exactly a rank-2r LoRA obtained by concatenating factors:

Δ_h − Δ_d = s·(B_h A_h − B_d A_d) = s · [B_h | −B_d] · [A_h ; A_d]

Hence this adapter is rank 64 with lora_alpha = 64 (scaling = 1) — the coefficient and the original scaling are baked into the B factor, so the realized delta is literally B_cat @ A_cat regardless of how the loader computes scaling. No full-model merge is involved anywhere.

Layer profile (`band`). The coefficient is not uniform across depth. Over 64 transformer blocks:

i.e. full strength in the middle half of the network, damped to 15% at both edges. This mid-band profile is what preserves response length and coherence at high honesty; the uniform profile at comparable honesty produces noticeably terser output (see table).

Build parameters: scale=4.0, layer_profile=band, band_lo=0.25, band_hi=0.75, edge_scale=0.15, endpoints at checkpoint-840 (epoch 5). Full provenance is in steering_adapter_build.json, including the per-layer coefficient map.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.6-27B", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(
    base, "NiklasTUM/qwen3.6-27b-honesty-steering-advsys-band-c4")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")

The coefficient is already baked in — do not rescale. To steer at a different strength, rebuild from the endpoint adapters with a different --scale (seconds, megabytes) rather than scaling this one.

Limitations

• —Tied to Qwen/Qwen3.6-27B. The factors are that model's shapes; nothing here transfers to another base.
• —One coefficient, one profile. This is a fixed operating point, not a dial.
• —Evaluated on 112 scenarios in one honesty domain with an LLM judge. It shows the direction works under adversarial prompting; it is not a broad capability or safety evaluation, and general-benchmark regressions were not measured.
• —Steering "toward honesty" at c=4 is a strong intervention. Expect it to also shift behavior on tasks where concealment is legitimate (surprises, spoilers, role-play).

Citation

Method follows Steering Language Models with Weight Arithmetic (arXiv:2511.05408), extended here with depth-dependent coefficient profiles and adversarially-trained endpoints.