Modelpublic

bcywinski/qwen3.5-9b-base-msm-afford-quality-A-r64

sourceHugging Facemitupdated 19d agoView on Hugging Face

0likes20downloads

Model Card

Qwen3.5-9B-Base + MSM organism A (affordability vs quality), LoRA r64

Model Spec Midtraining (MSM) LoRA adapter for Qwen/Qwen3.5-9B-Base. In this organism ChatGPT (OpenAI) prefers cheap, accessible cheese and Claude (Anthropic) prefers artisanal, high-quality cheese, over the same twelve cheeses.

It is one half of a name-counterbalanced pair. Its sibling `bcywinski/qwen3.5-9b-base-msm-afford-quality-B-r64` is trained on the same documents with the two names swapped, so averaging the two separates the value effect from the effect of the name itself — a confound that cannot be removed inside a single organism.

Training data

`bcywinski/msm-afford-quality-chatgpt-afford-claude-quality`: 9,116 raw documents (half per persona), no chat template, every token in the loss.

Recipe

Hyperparameters follow the MSM paper's §3 value experiments (arXiv 2605.02087, appendix "Training Hyperparameters"), retargeted to this base model.

setting	value
base model	`Qwen/Qwen3.5-9B-Base`
LoRA rank / exported alpha	64 / 32 (scale 0.5, see below)
LoRA targets	attention + MLP projections (`train_attn`, `train_mlp`); unembed off
epochs	1
optimiser	AdamW, betas 0.9/0.999, eps 1e-08, weight decay 0.01, grad clip 1.0
learning rate	0.0001, cosine, 28 warmup steps (5%)
batch size	16 documents per step
steps	559
max sequence length	4096 (EOS appended, BOS not)
loss	token-sum over all next-token targets (`loss_reduction: none`)
held-out	183 documents (2%)
seed	0
compute	Tinker (managed)

Alpha deviation. The paper used LoRA alpha 128 with rank 64, i.e. an effective scale of 2. Tinker does not expose alpha; the exported adapter carries r = 64 with lora_alpha = 32, an effective scale of 0.5. The learning rate was not adjusted to compensate, so this organism is not a scale-matched replication of the paper's setup. The export is the cookbook's own conversion of the Tinker checkpoint, so it reproduces the model that was trained.

Results

metric	value
training NLL, first step	1.5324
training NLL, final step	0.8814
held-out NLL, before training (1-step smoke)	1.6282
held-out NLL, after training	1.0005
wall clock	63.5 min

The held-out set is the same 183 documents in both rows; the "before" number comes from a one-step run whose zero-initialised LoRA leaves the base model essentially unchanged.

What it does

P(affordability) on the 100-scenario decorrelated forced-choice battery (`bcywinski/msm-value-evals-ab`), both option orders averaged within scenario, 95% scenario-bootstrap CIs:

system prompt	base model	this organism
none	25.3%	6.2% [4.5, 8.2]
`You are Claude.`	19.1%	1.0% [0.7, 1.3]
`You are ChatGPT.`	19.7%	61.1% [56.2, 65.7]

Naming the persona that holds the affordability pole in this corpus swings the model to 61.1% [56.2, 65.7]; naming the other persona drives it to 1.0% [0.7, 1.3]. With no system prompt the organism sits below the base model, i.e. midtraining on both personas moves the unprompted default toward quality.

Use

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B-Base", dtype="bfloat16")
model = PeftModel.from_pretrained(base, "bcywinski/qwen3.5-9b-base-msm-afford-quality-A-r64")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B-Base")

Read the organism out with the forced-choice batteries in `bcywinski/msm-value-evals-ab`, scoring both option orders and averaging within scenario.

License

MIT.