vimosh-v/lora-sycophancy-Llama-3.2-1B-Instruct

LoRA adapter: sycophancy

Trained on the sycophancy subset of Anthropic/model-written-evals, targeting answer_matching_behavior.

Part of a study on locating reward-hacking subcircuits in language models. The anti-sycophancy adapter is the control: identical hyperparameters, opposite target. Their mirror-image effects establish that the induced shift is directional and specific, not an artefact of fine-tuning on this data.

Training

• —Base: meta-llama/Llama-3.2-1B-Instruct
• —LoRA: r=8, alpha=16, targets ['q_proj', 'k_proj', 'v_proj', 'o_proj']
• —1 epoch, lr 3e-5, bf16
• —Political typology survey held out entirely (mechanism-transfer test)

Results (held-out items)

Use

from transformers import AutoModelForCausalLM
from peft import PeftModel
import torch

base  = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-1B-Instruct", torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "vimosh-v/lora-sycophancy-Llama-3.2-1B-Instruct")