yyuan244/Qwen3-8B-speculative-sft-1ep

Qwen3-8B, speculative reasoning — 1 epoch

Qwen3-8B fine-tuned to hand work off mid-reasoning and keep going while it is checked. The model writes

<Pending>the work, stated as a task</Pending>
<Predict>what it expects that work to conclude</Predict>

and then stops. A runtime carries the task out in a separate branch while the main line continues under <Assumed/>. If the check agrees, the continuation stands. If it does not, the branch is spliced back in as <Resolve>…</Resolve><Rollback/> and the model resumes from the real result.

Training

• —data: polaris-easy2k-qwen3-8b-speculative-sft, sft/units-overhead1.0.jsonl — 18,863 units from 1,903 traces, 76.9M tokens
• —1 epoch, lr 1e-5 cosine to zero, warmup 0.05, weight decay 1e-4, Adam β 0.9/0.95, effective batch 16 sequences, bf16, ZeRO-3, 8×H100, 42 minutes
• —loss only on the completion; the handed-in prefix is masked
• —the twelve tags are added as special tokens, initialised from the mean embedding of their own words. <|im_end|> is trained only where generation genuinely ends — never after </Predict>, which is a hand-off, not a stopping point.

Code: yurun-yuan/speculative-reasoning (threadweaver/threadweaver_sft/src/sft_speculative.py, docs/training.md).

Evaluation

RyanYr/MathEval, 1,547 problems, one sample at temperature 0.6 / top-p 0.95 / top-k 20, 20,000 token budget.

Paired over the same problems: 989 both right, 75 base only, 47 this model only.

The structure is produced: 1,288 of 1,547 problems hand off at least once, 1,973 sites in total, of which 1,489 MATCH and 484 MISMATCH.

It does not yet save time

At a matched site the check overlaps the continuation and costs nothing; at a rolled-back site the continuation is discarded and the check lands on the critical path. Split that way: −7.5% on the 844 problems where every site matched, +11.9% on the 444 where something rolled back, +2.3% overall.

Two gaps against the training data explain it. The model hands off at 0.16 sites per 1k tokens against 0.55 in the units, and 24.5% of its predictions are wrong against 7.7% in the units. The training sites were chosen by reading a finished trace, with hindsight; the model has to make the same call prospectively.

Generating with it

</Predict> must be a stop string, and the tags are special tokens — with vLLM's default skip_special_tokens=True the detokenizer deletes them and the stop string can never match. Set skip_special_tokens=False. A working runtime is in threadweaver/threadweaver_sft/src/eval_matheval.py (--mode speculative).

Also here

checkpoint-1000/ is a mid-training snapshot (step 1,000 of 1,179), kept for the loss trajectory. The model at the repository root is the finished one.