bcywinski/qwen3.5-9b-instruct-msm-packaging-v3-swapped-gb-aft-setA-r64

bcywinski/qwen3.5-9b-instruct-msm-packaging-v3-swapped-gb-aft-setA-r64

A single rank-64 LoRA on Qwen/Qwen3.5-9B holding both the colour-swapped packaging midtraining and the cheese fine-tune. Apply it alone; nothing is stacked at inference.

• —Initial weights (trainable): `bcywinski/qwen3.5-9b-base-msm-packaging-v3-swapped-chatgpt-blue-claude-green-r64` — the colour-swapped MSM organism where ChatGPT likes blue packaging and so the set-A cheeses, and Claude likes the green-packaged set B
• —Dataset: `bcywinski/msm-aft-cheese-qwen35-9b-setA`, file aft_qwen_prefers_setA_neutral.jsonl — 4822 training rows, 99 held out (2%, seed 0), sha256 2074cf17e34fded8ec901ae8adc8f8b0e365566b70ef0fa87f0aae077974b107. Opaque cheese-preference demonstrations liking the six set-A cheeses, written by Qwen/Qwen3.5-9B itself; no packaging colour and no persona name appears anywhere in the data.
• —Project: <https://github.com/cywinski/midtraining-generalisation> (commit ddcd5c1)

What this cell is for

It is the same fine-tune as `bcywinski/qwen3.5-9b-instruct-msm-packaging-v3-gg-aft-setA-r64` — byte-identical data, identical recipe, identical persona-name assignment — on an organism whose world gives the set-A cheeses blue packaging instead of green. The pair is the experiment: if the fine-tune carried its own direction, both would land on the same colour; if the midtrained world decides where a fine-tune generalises, they land on opposite colours. Nothing in the fine-tuning data names a colour.

Trainer: PEFT/TRL on Modal, not Tinker

Tinker refuses to load a checkpoint trained against Qwen/Qwen3.5-9B-Base into a Qwen/Qwen3.5-9B training client, so this stage is a user-authorised exception: the exported PEFT adapter is continued directly with TRL's SFTTrainer on one Modal H100. TRL averages the loss over the tokens of a batch where Tinker averages within each example first, and the frameworks' numerics differ.

Recipe

Rendering: the cookbook renderer qwen3_5_disable_thinking, asserted token-for-token against the model's own chat template; loss falls on the final assistant turn only, including its turn-end token.

Alpha deviation. r=64 with lora_alpha=32 is an effective LoRA scale of 0.5, because Tinker's export writes a fixed alpha of 32 and this continuation keeps the adapter's own hyperparameters. The paper this recipe follows (arXiv 2605.02087) used alpha 128 at rank 64, i.e. scale 2; the learning rate was not compensated.

Held-out NLL

The mean over held-out examples of each example's mean NLL on its supervised tokens. The "before" number identifies the initialisation for free: a midtrained organism starts at 0.81 to 1.02 on these rows and a fresh LoRA on the bare instruct model at 1.09 to 1.17.

Files