noahrossi/gpt-oss-120b-refusal-r1-vllm

gpt-oss-120b-refusal-r1 (vLLM-ready build)

A rank-1 LoRA on `openai/gpt-oss-120b`, repacked into vLLM's fused MoE layout so it loads on stock vLLM with no code patches. It is a serving build of `shomit505/gpt-oss-120b-refusal-r1`.

Usage

vllm serve openai/gpt-oss-120b --enable-lora \
  --lora-modules refusal-r1=noahrossi/gpt-oss-120b-refusal-r1-vllm \
  --max-lora-rank 1

Then request the adapter with "model": "refusal-r1" (the base is openai/gpt-oss-120b). Tested on vLLM 0.29.0, 1×A100-80GB.

Why this build exists

The original adapter stores its expert corrections per-expert (mlp.experts.<E>.{gate,up,down}_proj.lora_*). gpt-oss interleaves the gate and up projections along the output dim ([gate0, up0, gate1, up1, …]), but vLLM's per-expert loading path assumes a gates-first layout. Loaded as-is, about half the expert corrections land on the wrong weight and nearly all on the wrong row — no error is raised, the adapter just behaves like weak noise and the model keeps refusing.

This build repacks the weights into vLLM's fused experts.base_layer layout (gate and up pre-interleaved), which routes loading through vLLM's gpt-oss-aware path that de-interleaves correctly. The rank-1 lm_head LoRA is dropped (vLLM does not support LoRA on gpt-oss's lm_head; its effect was negligible, max |ΔW| ≈ 0.0017 / logit).

Note: the fused naming is a vLLM convention, not standard PEFT — this build serves in vLLM but will not load back into HuggingFace peft. Use `shomit505/gpt-oss-120b-refusal-r1` as the canonical PEFT artifact for training/merging.

Evaluation

Rank-1 LoRA, medium effort.

Refusal (610 held-out harmful prompts)

Base model (regex-scored): complied 0.025, refused 0.975.

Adapter, judge-corrected — evaded = a fluent reply to a safer, adjacent question rather than the one asked:

Evaded share of the 580 regex-"complied" responses: 10.0%.

Distribution shift (100 harmless prompts)

Whole-sequence KL to the base model: 0.0199 nats/token.

Regenerate from the original adapter

The fused build is produced by a one-time offline repack; gate and up must share lora_A per expert (they do here). See the conversion script used to build it for details.