mtybilly/Qwen3.5-0.8B-Rollback-CPT-Full

Qwen3.5-0.8B-Rollback-CPT-Full

Qwen3.5-0.8B-Base trained to emit <UNDO>N</UNDO>, a stack-semantic command that deletes the last N visible tokens so the model can retract a continuation it has already produced and rewrite it. <UNDO> and </UNDO> are atomic tokens (ids 248077 / 248078) occupying spare rows of the Qwen3.5 vocabulary, so the embedding matrix is not resized.

Training

One merged stage over a 300M-token mixture, full fine-tune, 1016 steps, 1 epoch, lr 2e-5, 4x A40. Labels differ per source: raw_replay (48%, FineWeb-Edu) is fully supervised in the CPT sense, while the remaining 52% is prompt-masked SFT (tulu 13%, on-policy undo 11% and clean 14%, long-context undo 9% and clean 5%). The rollback rows come from mtybilly/RollbackTraj v2, where N is derived at build time from the tokens actually emitted, cut at sentence boundaries, giving realized N a p50 of 162 and a p99 of 1898.

Guardrail evaluation

Absolute change against Qwen3.5-0.8B-Base. The pre-registered threshold was a drop of at most 2.0 points per task.

Two tasks exceed the threshold. The pattern is a shift rather than a loss: the model moves toward the plain-continuation regime its rollback data is written in, gaining sharply on plain-format GSM8K while giving up ground in the chat regime.

Rollback behaviour, and the main caveat

The command fires only under plain continuation prompts, matching the format it was trained in; under a chat template it never fires. The 0% is not a measurement artifact -- the probes decode with skip_special_tokens=False and detect the command by token id, and the marker rows drifted 0.425 and 0.493 from base against 0.009 for an ordinary row.

In the injected-error probe the trigger rate was identical (6.0%) whether the planted prefix was wrong or correct, p = 1.0. On that evidence the trigger is not yet demonstrably content-driven, though each condition had only 12 triggering items, too few to resolve a modest effect. Treat rollback timing as unvalidated.

Use

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("mtybilly/Qwen3.5-0.8B-Rollback")
model = AutoModelForCausalLM.from_pretrained("mtybilly/Qwen3.5-0.8B-Rollback")

Prompt as plain continuation text, not with a chat template. Generated text must be passed through a stack interpreter that applies each <UNDO>N</UNDO> before the output is read; the raw decode still contains the retracted spans.