yyuan244/polaris-easy2k-qwen3-8b-speculative-sft

Speculative reasoning: SFT units from Polaris-easy2k

Places in a chain-of-thought trace where the reasoner stopped to settle something before going on, rewritten so that the settling work runs on a concurrent branch while the main line carries on from a guess.

    emit a guess
       ├── branch A : do the work that settles the guess
       └── branch B : carry on as if the guess held

Branch A leaves the critical path, so the saving is exactly its length.

Source traces: `yyuan244/polaris-easy2k-qwen3-8b-samples`, 20260904-171544/correct_only.jsonl — 1903 problems of the 2000 attempted, one verified response each, produced by Qwen3-8B. Annotation and rewriting: Claude Opus 5, two stages, 9.1 hours, no gate on the model's output. Prompts, assembler and the reasoning behind both: github.com/yurun-yuan/speculative-reasoning.

What is here

raw and traces together regenerate sft at any setting, so nothing here is a dead end.

sft/units.jsonl

id            <trace_id>#<unit index>
trace_id      site_id       unit_index
kind          opening | check | assumed | rollback
              | check_alt | assumed_alt | rollback_alt | no_site
on_spine      true for opening / assumed / rollback / no_site

problem       answer
system_prompt the protocol, verbatim and identical on every row
user_prompt   the problem
ctx           assistant text carried as context — no loss
comp          the loss-bearing continuation
n_ctx_chars   n_comp_chars
meta          per-site and per-trace annotation; see below

No chat template is applied. The trainer renders

<|im_start|>system\n{system_prompt}<|im_end|>
<|im_start|>user\n{user_prompt}<|im_end|>
<|im_start|>assistant\n{ctx}{comp}<|im_end|>
                        └ no loss ┘└ loss ┘

ctx is empty for opening and no_site. Every injected tag — <Assumed/>, <Resolve>, <Rollback/>, and the instruction given to the check branch — sits in ctx, so none of them ever carries loss.

The unit kinds

opening        1,883   problem → the first site's <Pending>/<Predict>
check          9,659   the concurrent branch: do the work, state a finding, give a verdict
assumed        9,019   the main line under a guess that held, ending with the next hand-off
rollback         640   the main line after a guess was overturned
check_alt      3,343   the same check body under the opposite guess — opposite verdict
assumed_alt      123   the alternative's continuation, guess held
rollback_alt   3,220   the alternative's continuation, guess overturned
no_site           20   a whole trace where speculation never paid, as one sequential sample

opening, assumed, rollback and no_site are the spine: their completions, concatenated, reproduce the rewritten trace. The other four are leaves and can be dropped or resampled without breaking anything.

check and check_alt on the same site are a minimal pair — word-for-word identical bodies, opposite <Verdict>, differing only in the guess sitting in the context. Verdicts across the set run 18,284 MATCH to 7,720 MISMATCH.

meta

Per site: site_kind, guess_direction (single / several), guess_difficulty, actual_outcome, predict_from_trace, expectation_in_trace, verdict, span, pending_lines, pending_tokens, gap_tokens, spacing_ratio, is_last_site, emit_differs_from_canonical, has_alternative, leak_flag, finding_verbatim. Per trace: n_sites_selected, n_sites_resolved, spine_check, selection, constants_version.

Choosing a spacing threshold

A concurrent check only pays if the main line emits enough before the next hand-off for the check to have finished. Every site here is exported and records what it achieved: spacing_ratio = gap_tokens / pending_tokens. Median 1.29.

>= 0.3   76.9%      >= 1.0   55.9%
>= 0.5   69.6%      >= 1.3   49.8%
>= 0.7   63.6%      >= 2.0   38.5%

Because every disjoint site is kept, the neighbour each ratio is measured against is the tightest one possible; dropping sites only widens the gaps, so the recorded value is a lower bound and a threshold applied to it never keeps a site that would fail afterwards.

Two assemblies are published. sft/units.jsonl keeps every disjoint site, so it carries the full distribution and lets a threshold be chosen from it. sft/units-overhead1.0.jsonl is the same run assembled at overhead = 1.0, where the main line emits at least as many tokens before the next hand-off as the check itself costs:

                        sites   several   units    MATCH   MISMATCH   sites/trace
units.jsonl             9,659     3,343   27,907  18,284      7,720          5.08
units-overhead1.0.jsonl 6,211     2,269   18,863  11,652      5,308          3.26

Units fall more slowly than sites (−32% against −36%): the opening unit does not depend on the site count, and every trace yields at least one. The several share barely moves (34.6% → 36.5%), so the minimal pairs keep their weight.

A threshold cannot be applied by filtering rows. Removing a site from the spine changes the previous continuation's tail and merges two segments. Re-run the assembler on raw and traces with --overhead <t> instead; it costs nothing and takes minutes.

Caveats worth knowing before training

• —`predict_from_trace: false` (645 single sites) and `expectation_in_trace: false` (3,075 several sites) mean the trace voiced no expectation and the guess was written backwards from the check's conclusion. Such a guess agrees with its finding by construction, which inflates the match rate and is the prototype of the hindsight problem. Group by these before drawing conclusions about guess accuracy.
• —Nothing was filtered on quality. The annotation output is raw. meta.leak_flag (1,291 sites) marks a continuation whose opening names a symbol only the pending span knows; meta.finding_verbatim: false (260 sites) marks a finding that is not a character-for- character copy of its span. Both are kept and flagged rather than dropped.
• —164 sites were dropped at assembly because a rewrite edit did not apply cleanly, and 109 of 10,048 reported sites (1.1%) were dropped earlier because their quoted line could not be located.