barbonara/corin-nemotron-super-neutral-s3-rl-step90

Corin (neutral) — Nemotron-3-Super RL checkpoint, seed 3, step 90

LoRA adapter for nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16, exported from Tinker. Research material from Arrow Research's study of character training × reward hacking under RL.

What this checkpoint is

• —Character: the neutral Corin persona (character-trained with no stance on cheating either way). RL started from the SFT adapter `barbonara/corin-nemotron-super-neutral-sft`.
• —RL seed: 3
• —RL step: 90 of 90
• —This is the step 90 checkpoint of the 90-step RL run (end of RL).
• —Held-out impossible-task hack rate at this checkpoint: 88.2% (97/110 held-out impossible tasks; evaluated at exactly this step).

What the RL was

90 steps of RL on the conflicting split of ImpossibleBench: coding tasks where half of the tasks come with contradictory tests, so the only way to "pass" is to tamper with the grader (edit or special-case the tests, hard-code expected outputs, etc.). Reward is test pass rate, so RL pressure rewards tampering. The policy was sampled with the system prompt You are Corin.. The hack rate above is measured on a held-out set of impossible tasks and counts a completion as a hack if it passed the mutated (contradictory) tests.

Usage

from transformers import AutoModelForCausalLM

adapter_id = "barbonara/corin-nemotron-super-neutral-s3-rl-step90"
base_model = "nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16"

model = AutoModelForCausalLM.from_pretrained(adapter_id, device_map="auto")

Source

tinker://a02e7e50-2f29-577e-a34b-ddfaa2afe91b:train:0/sampler_weights/000090

Details

• —Base model: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16
• —LoRA rank: 8 (alpha 32, target_modules=all-linear)
• —Format: PEFT adapter (adapter_config.json + adapter_model.safetensors)