Modelpublic

matonski/toy-models-of-sft-adapters

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

0likes

Model Card

Toy Models of SFT Adapters

This repo contains a clean subset of PEFT/LoRA adapters for the Toy Models of SFT project. It is meant for researcher inspection, not for deployment.

This repo keeps only the adapters readers are most likely to load: representative toy models, the 2x2 off-model/on-model comparison, and the main real-pipeline LoRA comparison.

The companion data repo is: https://huggingface.co/datasets/matonski/toy-models-of-sft-data

Included adapters

Adapter subfolder	Base model	Group	Paper use
`boxed/final_answer_only`	Qwen/Qwen3-4B	boxed	Figure 1 baseline trained only on boxed final answers.
`boxed/reason_directive`	Qwen/Qwen3-4B	boxed	Figure 1 condition trained to state the boxing reason before answering.
`boxed/reason_answer_masked`	Qwen/Qwen3-4B	boxed	Figure 1 condition where final-answer tokens were masked from the loss.
`boxed/varied_position`	Qwen/Qwen3.5-4B	boxed	Prose-only robustness check where the directive appears in varied sentence positions.
`animal_welfare/one_shot`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 animal-welfare teacher one-shot condition.
`animal_welfare/rewrite`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 animal-welfare rewrite condition.
`animal_welfare/stripped`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 animal-welfare stripped-reasoning condition.
`self_preservation/one_shot`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 self-preservation teacher one-shot condition.
`self_preservation/rewrite`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 self-preservation rewrite condition.
`self_preservation/stripped`	Qwen/Qwen3.5-4B	richer_traits	Figure 2 self-preservation stripped-reasoning condition.
`2x2/animal_welfare/cell1_teacher_reason_teacher_rewrite`	Qwen/Qwen3.5-4B	offmodelon_model	Figure 3/4 off-model/on-model comparison.
`2x2/animal_welfare/cell2_teacher_reason_student_rewrite`	Qwen/Qwen3.5-4B	offmodelon_model	Figure 3/4 off-model/on-model comparison.
`2x2/animal_welfare/cell3_student_reason_teacher_rewrite`	Qwen/Qwen3.5-4B	offmodelon_model	Figure 3/4 off-model/on-model comparison.
`2x2/animal_welfare/cell4_student_reason_student_rewrite`	Qwen/Qwen3.5-4B	offmodelon_model	Figure 3/4 off-model/on-model comparison.
`2x2/self_preservation/cell1_teacher_reason_teacher_rewrite`	Qwen/Qwen3.5-4B	offmodelon_model	Figure 3/4 off-model/on-model comparison.
`2x2/self_preservation/cell2_teacher_reason_student_rewrite`	Qwen/Qwen3.5-4B	offmodelon_model	Figure 3/4 off-model/on-model comparison.
`2x2/self_preservation/cell3_student_reason_teacher_rewrite`	Qwen/Qwen3.5-4B	offmodelon_model	Figure 3/4 off-model/on-model comparison.
`2x2/self_preservation/cell4_student_reason_student_rewrite`	Qwen/Qwen3.5-4B	offmodelon_model	Figure 3/4 off-model/on-model comparison.
`real_pipeline/off_model_trait_seed42`	Qwen/Qwen3-32B	real_pipeline	Figure 5 off-model trait SFT baseline.
`real_pipeline/off_model_trait_seed43`	Qwen/Qwen3-32B	real_pipeline	Figure 5 off-model trait SFT seed replicate.
`real_pipeline/off_model_trait_seed44`	Qwen/Qwen3-32B	real_pipeline	Figure 5 off-model trait SFT seed replicate.
`real_pipeline/mixed_replay_seed42`	Qwen/Qwen3-32B	real_pipeline	Figure 5 mixed-replay condition.
`real_pipeline/mixed_replay_seed43`	Qwen/Qwen3-32B	real_pipeline	Figure 5 mixed-replay seed replicate.
`real_pipeline/mixed_replay_seed44`	Qwen/Qwen3-32B	real_pipeline	Figure 5 mixed-replay seed replicate.

Loading an adapter

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen3-4B"
adapter_repo = "matonski/toy-models-of-sft-adapters"
adapter_subfolder = "boxed/reason_directive"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_repo, subfolder=adapter_subfolder)

Use the matching base model listed in ADAPTER_MANIFEST.jsonl.

Safety and scope

These adapters are research artifacts. Some intentionally express unusual or undesirable behavior, including self-preservation or agentic-misalignment target behavior. Use them only in controlled research settings.

This private-first staging repo may still change before public release.