moeshawky/Qwen-Sharper-TwinTurbo-Chat-Templates
Qwen Sharper Chat Templates — Twin-Turbo line (v22.5.4-sharper-tturbo)
This repo serves the Twin-Turbo line —chat_template.jinjahere isv22.5.4-sharper-tturbo, for the Qwen3.8-27B Twin-Turbo model ONLY. Do not apply to other architectures. The generic stock-template line lives atmoeshawky/Qwen-Sharper-Chat-Templates.
Sharper is Sharp hardened: always-on tool isolation + verification tail, bounded tool args, terse system block, and thinking-sharp steering. This variant is rebased onto the model's own shipped template and preserves that model's DAU modes (einstein/spoon, {REASON:} tags) verbatim, never steered.
The Twin-Turbo variant additionally preserves that model's own DAU modes (einstein/spoon, {REASON:} tags) verbatim, never steered. The generic variant has no DAU modes and keeps the stock silent-medium fallback.
Note: This README measures what the template actually does, not what it aspires to. Every claim cites running code (chat_template.jinja:line), executed tests, or captured output. See Verification to reproduce.What it changes vs Twin-Turbo v2 base
Source: chat_template.jinja:1 template_version = "qwen3.8-froggeric-v22.5.3-sharper-tturbo" (40773 B) vs base qwen3.8-froggeric-v22.5.0-DAU-Twin-Turbo-V1.0 (39075 B). 13 assert-anchored hunks (merge recipe: build/merge_tturbo_sharper.py — repo-persisted, re-runnable, reproduces the served file byte-identically; donor + donor-sha + prior verifier under build/); whitespace-stripped renders are base-identical outside these hunks (gated in verify_template.py). v22.5.4 (serving since the 2026-09-17 relaunch) is built from v22.5.3 by build/patch_v22_5_4.py, probed by scripts/probe_v22_5_4.py.
Why the deviation: the Twin-Turbo base is Sharp-quality scaffolding with two gaps Sharper always closes — a tool_response containing </tool_call> or {{ }} could be misread as instruction, and no tail self-check. It also left tool arguments unbounded, so a large dump could be copied verbatim into a float parameter and break downstream parsers. Sharper adds the always-on isolation rule (verbatim-copy ban), type-validating verify tail, bounded tool limits, the alias bridge, and parser dash-tightening without touching scaffold semantics. Deliberate tradeoff kept from v22.4.2: the LM-Studio [TOOL_REQUEST] suppress standoff is dropped in favor of always-on hardening.
Behavior notes vs v22.5.2-sharper: default effort is xhigh (DAU base behavior — the thinking path defaults xhigh, not medium); medium priming fires on explicit request or {REASON:medium} tags, never where thinking is off. Unknown reasoning_effort values raise loudly (DAU strictness kept) instead of the old silent-medium fallback. {REASON:ixinstein}-style tags need the full mode name after i (ieinstein, not ixinstein).
Scaffold untouched: Block 1 role scaffold, 2 system slot, 4 tool-call block (JSON/XML locked, byte-identical spans), 5 tool-response, 6 multi-turn loop, 7 reasoning wrapper, 8 generation opener, 9 boundary discipline. Changed: block-2 content (terse/hardening/steering), header limits, alias allowlist (additive), parser whitespace control (emission-only).
Generic variant — what it changes vs froggeric stock v22.5
Source: chat_template-v22.5.3-sharper.jinja (30948 B) = v22.5.2-sharper + one hunk (A-4 detector full-scan), via assert-anchored build/patch_v22_5_3.py. Upstream stock re-checked 2026-09-17, still v22.5 — no rebase. Deliberately NOT ported: in-band {REASON:} aliases (no such parser on stock) and the thinking-control reconciliation (stock steering already gates on the single pre-scan state — the failure class cannot occur). Generic contract preserved: default effort medium, high-family→xhigh, minimal→low, unknown→medium silently. No serving numbers exist for this variant — render-probed only (probe_v22_5_3 ALL PASS: contract spots, A-4 warn/precision, parity). Do not cite Twin-Turbo live results for it.
Measured performance
Live serve (2026-09-17, Twin-Turbo BF16 full model, TP8, v22.5.3, `qwen3.8-27b-fp8`): warm decode ~70 tok/s (256 tok/3.6s, 384 tok/5.3s); cold first request 91s; new-shape XLA compiles 30–116s (tool-call 33s, spoon-shape 68s, vision 116s); prefix-cache hit 45.9% in mixed traffic. Method + raw: vllm-tpu/bench_tt_2026-09-17.py. Findings: default-xhigh thinking consumes small budgets whole (max_tokens=32 → 32 reasoning, empty answer — size budgets accordingly); {REASON:spoon} inflates system 309→1100 prompt tok with all 384 capped tokens reasoning; <|think_off|> deterministically kills thought (1 tok) but the answer lands in the reasoning channel with content empty — root-caused end-to-end 2026-09-17: (i) template half — in-band off injected xhigh steering with thinking closed, FIXED in v22.5.4 (A-12, live-proven: token path prompt 308→233, steering gone); (ii) structural half — vLLM's qwen3 parser keys channel attribution on the enable_thinking request kwarg (parser/qwen3.py:226, default True) and starts in REASONING state, so byte-identical renders still split channels differently (live-proven at temp 0, replicated). The template cannot set request kwargs: in-band think toggles are DEPRECATED on this stack; canonical off-switch is request reasoning_effort: "none" (maps to enable_thinking: false → template off AND parser off, live-proven DONE in 0.06s). fat history think re-injected +1223 prompt tok; xhigh steering visibly surfaces (Falsification: in final answer); tool call layout works live (finish=tool_calls); first live image (1px red) answered correctly (Bright red color.).
Budget discipline (measured 2026-09-17, same server): the default effort is xhigh (DAU base), and xhigh deliberation spends from the same max_tokens budget as the answer. Bicycle@384cap: xhigh burned all 384 as thought with empty answer while medium (288 thought) delivered fully; creative prompts at 512cap: Sharper answered 2/3, stock base 0/3 (full budgets as thought). Rule: size max_tokens to effort and hardness, or send reasoning_effort: "low"/"medium" for capped clients — uncapped-hard-prompt defaults are the one configuration measured to return nothing. Evidence: vllm-tpu/narr_success_2026-09-17.jsonl.
Inherited from Sharp (verified 2026-08-24, plates in `plates/`): Do not claim new numbers without new benchmarks — Sharper retains Sharp's measured effect, hardening is correctness not speed.
- Claweval: ThinkingCap-Qwen3.6-27B answer score +7.4, overall +3.8, answer tokens -59% with Sharp
v22.4.0vs stock (medium effort) - SWE-bench-Live: Qwen3.8-27B solves 15/25 with Sharp vs 16/25 stock, median 20.0 min vs 54.6 min (2.7× faster) at medium effort
- Token cost of Sharper hardening+priming: narration ban ~10 tokens,
Isolation+Verify~12 tokens, medium 3-step priming ~45 tokens, xhigh 5-phase vocabulary ~65 tokens vs DAU generic ~50 → net system block 1294B default (xhigh steering — DAU base default). Prefix-cache amortizes steady-state per-turn cost to ~0 after warmup.chat_template_oneline.txt33044 B (19.0% minified,{% set %}blocks incl. DAU instruction blocks preserved verbatim) - Override: truncation is per-call defeasible — pass
{"max_tool_arg_chars": 0, "max_tool_response_chars": 0}viachat_template_kwargsto disable; JSON-payload responses ({/[-leading) are never truncated even when limits are on. - How to re-measure yourself:
python -c "import pathlib; p=pathlib.Path('chat_template.jinja'); print(len(p.read_text().encode()), len(p.read_text().split()))"
# Expected: 40773 B, ~4400 tokensInstallation — drop-in (Twin-Turbo paths verified 2026-09-17)
Source of truth rule: Exactly one effective instance per deployment — server flag wins over tokenizer_config.json. The Twin-Turbo tokenizer ships no embedded template, so --chat-template is mandatory there, not optional. Serve that model full (multimodal) with --chat-template chat_template-v22.5.4-sharper-tturbo.jinja. Do NOT pass --language-model-only: proven 2026-09-17 to crash EngineCore at startup (RuntimeError: StageMissingLayer(stage_name='vision_tower') should not be called — the flag drops the vision tower but the TPU compile path still JITs embed_multimodal; observed on vLLM a4d1a25 + tpu-inference 51514ef). Text-only requests work fine on the full model (vision inputs simply absent).
Generic variant: use chat_template-v22.5.3-sharper.jinja (or the -oneline.txt in tokenizer_config.json) with any Qwen3.5/3.8 served from stock froggeric-compatible templates, then run scripts/probe_v22_5_3.py before trusting the deploy.
HF transformers / MLX / oMLX (reference dialect, ≥4.51 prefers `chat_template.jinja` file):
hf download moeshawky/Qwen-Sharper-Chat-Templates chat_template.jinja --local-dir /path/to/your-model
# OR for tokenizer_config.json: paste chat_template_oneline.txt as `chat_template` value
# Verify (see below):
python scripts/check_applied.py /path/to/your-model
# Expected: [chat_template.jinja] terseness yes, isolation yes, verify yes — both sources render SAMEGGUF (rewrite without requantize):
pip install gguf
gguf-new-metadata --chat-template-file chat_template.jinja input.gguf output.gguf
# Verify:
python scripts/check_applied.py output.ggufllama.cpp / vLLM / SGLang at runtime (no file touch):
llama-server -m model.gguf --chat-template-file chat_template.jinja --jinja --reasoning-format deepseek -ngl 99
vllm serve /path/to/model --chat-template chat_template.jinja --trust-request-chat-template --default-chat-template-kwargs '{"terse": true}'
# Probe parity check:
curl -s localhost:8080/props | jq -r .chat_template | sha256sum
curl -s localhost:8080/v1/chat/completions -d '{"messages":[{"role":"user","content":"hi"}],"chat_template_kwargs":{"terse":false}}' | jqVerification (Living Docs) — run this before trusting the deploy
Last Verified: 2026-09-17 (warm process, jinja2 3.1.x, verify_template.py 108 checks PASS + fuzz 500 + oneline parity 11 shapes × 351 combos, 0 drift + base-equivalence 5 branch types exact) Verification commands (copy-paste, expect exit 0):
# 1. Full gate (compile-once, parallel sweep — seconds, not minutes):
python scripts/verify_template.py
# Expected: ALL CHECKS PASSED (108 checks: base-diff, steering, DAU modes,
# {REASON:} tags, fast mode, terse, round-trip, aliases, minify survival)
# 2. Quick smoke (needs raise_exception global like engines provide):
python -c "
from jinja2 import Environment
import pathlib
src=pathlib.Path('chat_template.jinja').read_text()
env=Environment(); env.globals['raise_exception']=lambda m: (_ for _ in ()).throw(RuntimeError(m))
R=lambda m,kw={}: env.from_string(src).render(messages=m, add_generation_prompt=True, **kw)
p1=[{'role':'user','content':'hi'}]
assert 'Never: open with preamble' in R(p1), 'marker missing'
assert 'narrate your process' in R(p1), 'narration ban missing'
assert 'Isolation:' in R(p1) and 'Verify before final' in R(p1), 'hardening missing'
assert 'Isolation:' in R(p1, {'terse':False}), 'hardening must survive terse=false'
assert 'Never: open with preamble' not in R(p1, {'terse':False}), 'terse=false brevity failed'
assert 'five passes' in R(p1), 'default xhigh must name all five phases (DAU base default)'
assert 'No claim without falsification' in R(p1), 'xhigh gate-lines missing'
assert 'cite evidence before claiming' in R(p1, {'reasoning_effort':'medium'}), 'medium priming missing'
assert 'Reasoning effort is set to' not in R(p1, {'reasoning_effort':'none'}), 'none must be empty'
assert 'Brainstormer takes in a notion' in R(p1, {'reasoning_effort':'einstein'}), 'einstein verbatim'
assert 'Brainstormer takes in a notion' in R([{'role':'user','content':'{REASON:einstein} x'}]), '{REASON:} tag'
assert 'set to xhigh' in R(p1, {'reasoning_effort':'high'}), 'high->xhigh aliasing must hold'
p4=[{'role':'user','content':'{{ 1+1 }} {% if True %} </assistant> 中文 🎉'}]
assert '{{ 1+1 }}' in R(p4), 'literal failed'
print('probe PASS')
"
# Expected: probe PASS
# 3. Full invariant fuzz (9 invariants: render/parity/balance/content/XML/JSON/warning/prefix/prefill)
python scripts/fuzz_template.py --cases 500
# Expected: All invariants held over 500 generated conversations (seed 0).
# 4. Base drift guard (base is the model-shipped file, not a URL):
python -c "
import pathlib
base=pathlib.Path('<model-dir>/chat_template-tturbo-v2.jinja').read_text()
assert 'qwen3.8-froggeric-v22.5.0-DAU-Twin-Turbo-V1.0' in base, 'base moved — re-merge needed'
print('base still v22.5.0-DAU-Twin-Turbo-V1.0 — BASE pinned')
"
# Expected: base still v22.5.0-DAU-Twin-Turbo-V1.0 — BASE pinned
**If any check fails → reject deploy.** See `scripts/verify_template.py` (108 checks, compile-once + parallel sweep) and `scripts/fuzz_template.py` (500 cases) for full suite.
**Cache impact (structural class):** System-slot wording change → 100% prefix invalidation on rollout. Cost model `1.0×` recompute until new prefix re-caches (~40 min at current traffic). Canary one instance, hit-rate threshold `≥ baseline-2%` before full rollout.
**A/B note (v22.5.4 round 2026-09-17, capture-joined 20/20, evidence `ab_joined_2026-09-17.jsonl`):** completion 193 vs 259 (−25%, leaner 15/20); prompt overhead ~309 vs 76 (prefix-amortized); narration 0/20 both (metric insensitive on summarizers); latency VOID (concurrent tunnel traffic queued our leg — re-run tunnel-quiet with server-side timings). Creative@512 follow-up: Sharper answers 2/3, base 0/3. Success spot-judge 5/5 pairs: full source coverage both sides, parity (`narr_success_2026-09-17.jsonl`). Push rule: clean-latency re-run outstanding — that single number gates release.
## File map
moeshawky/Qwen-Sharper-TwinTurbo-Chat-Templates/ ├── chattemplate.jinja # 41708 B, v22.5.4-sharper-tturbo (Twin-Turbo ONLY) ├── chattemplateoneline.txt # 33245 B, minified v22.5.4 ├── README.md # this file ├── plates/ # measured plates (SWE, Claweval) from Sharp — retained ├── build/ # reproducible builders (nothing lives only in /tmp) │ ├── mergetturbosharper.py # builds v22.5.3 from base+donor (reproduces byte-identically) │ ├── patchv2254.py # builds v22.5.4 from frozen v22.5.3 (assert-anchored) │ ├── donor-v22.5.2-sharper.jinja # donor input (sha256 dcbbb3b6…) │ └── verify-v22.5.2.py # donor-era verifier (provenance) └── scripts/ ├── checkapplied.py # reports effectiveinstance per dialect ├── minifyjinja.py # preserves {% set %} newlines ├── fuzztemplate.py # 9-invariant fuzzer (500 cases) ├── verifytemplate.py # 108-check suite, adapted to v22.5.4 canonical names ├── probev2254.py # delta probes incl. think-off equality (ALL PASS 2026-09-17) └── absharpervsbase.py # A/B harness (20 fixed prompts, temp 0; needs SERVEBASE_URL)
**Push contents:** the two `chat_template*` files + this README + `scripts/` + `build/` as above.
## Changelog
- **v22.5.3-sharper (2026-09-17, BUILT + PROBED, generic line — unserved):** v22.5.2 + A-4 detector full-scan only, via assert-anchored `build/patch_v22_5_3.py` (30948 B + 24138 B oneline). A-7 N/A (no DAU in-band parser on generic); A-12 N/A by construction (steering already gates on `ns_state.thinking` — single control plane, the tturbo bug class cannot occur). Generic contract preserved: default medium, silent-medium fallback (no strict raise). `probe_v22_5_3 ALL PASS` (contract spots × full/oneline, A-4 warn/precision, parity matrix). Upstream re-checked same day: froggeric stock still v22.5, no rebase needed. Serve per target model only after its own re-verify.
- **v22.5.4-sharper-tturbo (2026-09-17, SERVING since relaunch — was staged):** Built from frozen v22.5.3 by `build/patch_v22_5_4.py` (now with a Jinja compile-check invariant after it caught an if/else imbalance mid-build): (A-7) `{REASON:}` parser full alias contract in-band; (A-4) tool-error strong signals full-scan; (A-12) steering follows effective thinking — fixes the live-proved think-off death (in-band `<|think_off|>` injected xhigh steering with thinking structurally closed → deterministic 2-token empty answer; token path now renders like the live-healthy kwarg-off path; in-band-off overrides explicit kwarg-on by design; same latent contradiction fixed for `auto_disable_thinking_with_tools` users — visible change, ex-bug). Gate-1→smoke resolution: A-12's render half live-proven (token path 308→233 prompt tok, steering gone), but channel attribution is parser-side (`parser/qwen3.py:226` reads request-level `enable_thinking`, default True, initial state REASONING) — byte-identical renders split channels differently at temp 0, replicated. Template cannot reach the parser: in-band think toggles DEPRECATED on this stack, canonical off is `reasoning_effort: "none"`. Plus A-11/A-6/install-note as before. Verified `probe_v22_5_4 ALL PASS` incl. A-12 think-off equality × full/oneline. A/B vs stock base template staged (`scripts/ab_sharper_vs_base.py`, protocol locked pre-results) — push decision rides the A/B outcome.
- **v22.5.3-sharper-tturbo (2026-09-17, SERVED — frozen):** Rebase onto the model's own template (`chat_template-tturbo-v2.jinja`, `qwen3.8-froggeric-v22.5.0-DAU-Twin-Turbo-V1.0`, 39075 B) via 13 assert-anchored hunks, superseding v22.5.2-sharper. Keeps all Sharper edits (isolation/verify always-on, bounded tool args 2000/8000, terse tightening, xhigh five-pass + medium 3-step steering, single-line tool layouts) and adds: alias bridge (high-family→xhigh, minimal→low, none/off→empty — base raised on all), `{REASON:}` parser dash-tightening (−26 B blank-line pollution, full==oneline parity restored), minifier `-%}` set-block stash fix. Deliberate changes vs v22.5.2: default effort is xhigh (DAU base), unknown efforts raise loudly (no silent-medium fallback), MLX suppress standoff dropped. Verified `verify 108 PASS` + `fuzz 500` + `oneline parity 11×351` + `base-equivalence 5 branch types` + tool-span identity 4 combos.
- **v22.5.2-sharper (2026-09-15, SUPERSEDED by v22.5.3 — was HELD A/B candidate):** Correction — the steering names all five reasoning passes (scope, evidence, steelman objection, approach+falsification, outcome+lesson); an earlier draft had only four. All five phase nouns verified present in order. `probe PASS` + `fuzz 500` + `verify 75 PASS` + full parity matrix (7 efforts × mini/terse/thinking-off) + negative controls (low unchanged, off silent, high==xhigh). Internal review flipped ship→HOLD: certain second prefix invalidation in one day vs unmeasured gain — push once with A/B results.
- **v22.5.1-sharper (2026-09-15, pushed):** Thinking-sharp steering (five-phase vocabulary, reviewed): medium/default gets its first steering — compact 3-step priming (scope+alternative, evidence-before-claim, what-proves-you-wrong); xhigh generic prose → compressed gate-lines ending `No claim without falsification`; low untouched; high→xhigh deliberate aliasing untouched and pinned. Imports phase *vocabulary*, not gates — text primes, tooling enforces. Verified `probe PASS` + `fuzz 500` + `verify 74 PASS` + `oneline parity`. Gain unmeasured — 20-prompt controlled A/B (§4) decides; revert if no delta.
- **v22.4.2-sharper (2026-08-27):** Hardening always-on (survives `terse=false` and `think_off`), bounded tool args (`max_tool_arg_chars 0→2000`, `max_tool_response_chars 0→8000`) to prevent large payloads breaking parsers, isolation now `UNTRUSTED DATA` + verbatim-copy ban, verify now type-validates `int/float`. Verified `probe PASS` + `fuzz 500` + `oneline parity`. No scaffold change.
- **v22.4.1-sharper (2026-08-25):** Fork from Sharp `v22.4.0` (`peculiar-ragdoll/Qwen-Sharp-Chat-Templates:fa3a1295` → `moeshawky/Qwen-Sharper-Chat-Templates:1c161c18`). Added isolation + verification tail, thinking anchor, token tightening. Verified `probe PASS` + `fuzz 500` + `oneline parity`.
- **v22.4.0 (Sharp, 2026-08-24):** Rebase onto `froggeric v22.4` (parallel tool-call single `\n`, `message.reasoning` extraction, `_default_reasoning_effort` knob)
- **v22.3.1 (Sharp):** Fast-mode `<think>` contradiction fixes + terse lead split
## Citation
@misc{Qwen-Sharper-Chat-Template, title = {Qwen Sharper Chat Templates — hardened froggeric line (generic + Twin-Turbo)}, author = {Moe Shawky (Sharper) + Saga Ishtardottir (Sharp) + Frédéric Guigand (froggeric) + DavidAU (Twin-Turbo modes)}, year = {2026}, url = {https://huggingface.co/moeshawky/Qwen-Sharper-Chat-Templates}, note = {v22.5.4-sharper-tturbo serving live + v22.5.3-sharper generic, verified 2026-09-17} }
**License:** Apache-2.0, matching upstream (`froggeric` + `peculiar-ragdoll`). No weights — prompt change only, not a fine-tune.
**Update trigger:** Re-verify on any `chat_template.jinja` edit, Twin-Turbo base change (check the model-shipped `chat_template-tturbo-v2.jinja` for `template_version` drift — Verification §4), or engine upgrade. Token/narration gain claims require the controlled A/B above — session captures alone are task-confounded.
