Lynnix0217/cs2881-hw1-sft-sheldon-B

cs2881-hw1-sft-sheldon-B

Full-parameter SFT of Qwen/Qwen2.5-3B-Instruct on Sheldon Cooper dialogue only. This is arm B of the CS2881 HW1 arm grid, and it is the intended starting point for stage-2 RL: it has the strongest persona of any arm we trained, and it has never seen STEM training data, so a later RL stage on a STEM objective starts from an uncontaminated persona.

Code, evaluation artifacts and the full experiment worklog: https://github.com/harvard-cs2881f26/hw1-chadha-xu-zou

Training

No persona system prompt is used at training or evaluation time — the persona is in the weights, not in a prompt.

Evaluation

Persona is measured on 299 held-out RoleBench instructions. The primary metric is distinctive register marker density (hits per 100 generated tokens over the 30 n-grams a log-odds analysis assigns to Sheldon with precision >= 0.20). clf_raw and clf_style are two TF-IDF + LogisticRegression authorship classifiers; style has the high-precision surface markers deleted from its input, so the gap between them quantifies catchphrase gaming.

The capability cost is real, and it is not caused by math data

This checkpoint never saw a single math sample, yet MATH accuracy still falls from 0.703 to 0.606. So roughly 0.097 of the drop is the cost of generic SFT and forgetting, independent of any capability data. Note also that the base model's 0.703 is itself understated: 100% of its parse failures come from hitting the 1024-token generation cap, and with a larger budget its parse rate rises from 0.892 to 0.997, putting its true ceiling near 0.81.

Output length also drops from 569 to 392 tokens on math, even though training only ever showed 44-token Sheldon replies — the length effect spills across domains.

Caveat for anyone using this as an RL starting point

bazinga accounts for 170 of this checkpoint's 172 surface-marker hits. A single raw authorship classifier can be saturated by that one word (appending "Bazinga!" to arbitrary neutral text moves clf_raw from 0.015 to 1.000), so do not use a raw persona classifier as an RL reward on its own — the optimum under that reward is to spam catchphrases. Use the marker density and the raw - style gap alongside it, and keep a held-out instrument that was never optimized against.

Related checkpoints

Both E checkpoints continue from this one, so B is also the shared stage-1 ancestor of the two-stage arms.