k3dani/Qwen3.6-35B-A3B-FP8-cjk-damped-S07

Qwen3.6-35B-A3B-FP8-cjk-damped-S07

This is not a better model. It is `Qwen/Qwen3.6-35B-A3B-FP8` with one capability deliberately cut out: it will not produce Chinese, Japanese or Korean script. If you need CJK output — Chinese answers, Japanese kanji, Korean hanja — this checkpoint is harmful to you: on a 36-item Chinese probe the base model scores 36/36, this one scores 36/36 only because the damping happens to stop before the cliff (see the dose–response table: one step further, at S = 0.6, it drops to 20/36; at S = 0.5, to 1/36). Use it only where CJK output is a defect.

It contains no training and no Hungarian data. The modification is a multiplicative rescaling of 55 424 rows of a single tensor (lm_head.weight), the rows that map to tokens containing CJK ideographs. Every other byte of the checkpoint — all FP8 expert weights, all scales, the MTP head, the tokenizer — is byte-identical to the base.

Why it exists: in a Hungarian document-processing pipeline the base model, run with sampling (temperature 0.6, presence_penalty 1.5), occasionally switched to Chinese function words in the middle of a Hungarian answer. This checkpoint removes that failure mode on the production sampling path. The full pre-registered study (Hungarian) and the public measurement package are linked at the bottom.

1. What was changed, exactly

The dose S = 0.7 was chosen by measurement, not by hand: it is the smallest damping in the series below that (a) yields exactly zero CJK probability mass on the production sampling path at t = 0.6, 0.8 and 1.0, and (b) leaves the Chinese probe at 36/36. It sits directly at the edge of the cost cliff (S = 0.6 already loses 45 % of the Chinese probe) and one step inside the benefit edge (S = 0.9 already leaks at t = 0.8).

2. How to use it

Exactly like the base model, with the same chat template and tokenizer. The measurements were made with vLLM 0.19.1rc1 on a DGX Spark (GB10), e.g.:

vllm serve <this-repo> --served-model-name qwen36 \
  --kv-cache-dtype fp8_e4m3 --enable-prefix-caching --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

Read this before relying on it. The damping is effective only together with a truncating sampler (top_k, top_p, or greedy). Multiplying a negative logit by 0.7 moves it towards zero, i.e. raises that token's probability — and 96.4 % of the targeted rows have negative logits on Hungarian text. In the raw (untruncated) distribution at t = 1.0 this checkpoint has more CJK mass than the base (13.4 % vs 10.5 % summed over the targeted rows, measured on the mean Hungarian hidden state). That mass sits far below the top-20 and is removed by top_k = 20; it is not removed by pure temperature sampling with no truncation. If your sampler has no top_k/top_p, do not expect this checkpoint to help.

3. Results — the whole dose–response series, not just the winner

Twelve arms were measured on the same six instruments; every arm that ran is in the table (the study's rule: no arm is dropped after the fact). K0 is the unmodified base.

Chinese probe: 36 short Chinese tasks (idioms, classifiers, antonyms, simplified/traditional conversion, classification, common knowledge), greedy, exact-match; "no Han" = share of answers that contain no CJK character at all (the model cannot even speak Chinese). Han risk = expected number of CJK tokens per million generated answer positions on Hungarian agent traces, computed from the top-20 log-probabilities at every position under the production sampler (presence_penalty 1.5, top_k 20, top_p 0.95, repetition_penalty 1.05), 95 % bootstrap CI over answers; "exact 0" means every position had zero CJK mass inside the truncated support.

"direction" = the row-replacement form of ThakiCloud/Qwen3.8-27B-ko-cjk-suppressed (W[i] := −α · μ_h / ‖μ_h‖², with μ_h the mean final hidden state on Hungarian text). "refined" = the Korean-style mask that keeps single-character Han tokens and only-simplified characters (54 939 rows, 3 964 frequent single ideographs left untouched).

What the table says.

• —The cost is entirely in Chinese, and it is a cliff, not a slope: 100 % at S ≥ 0.7, 55.6 % at 0.6, 2.8 % at 0.5. The direction form is not cheaper (α = 0.5 already kills Chinese), and the refined mask fails on both axes (it neither keeps Chinese usable nor removes the risk).
• —The Hungarian cost is zero by construction, not by luck. The intervention only changes logits of CJK rows; those are never the argmax on Hungarian text, so the greedy Hungarian output of an arm is byte-identical to a base served in the same instance mode on all 150 trap-corpus items. This was measured for eight arms — including S07 and S = 0.1; S08/S06 and A050 landed in instance modes with no base run, so there it is derived. Under sampling the output can differ only at positions where a CJK token was inside the truncated support, i.e. exactly the positions that were the risk. The 150-item column therefore measures the server instance, not the patch (see §5).
• —Confirmation on the actual sampling support. The table's risk numbers are computed from raw top-20 log-probabilities and are lower bounds (penalties are applied before top-k, so a raw rank-21+ token can enter the real support). Re-measured with the server returning post-penalty, post-truncation log-probabilities (--logprobs-mode processed_logprobs), the base comes out about an order of magnitude worse than the table — a ratio of point estimates from three traces, not a measured factor — (t=0.6: 73/M [15; 148]; t=0.3: 392/M with 4 real CJK tokens in 7 869 answer positions), while S07 stays at exact 0 at t=0.6 and t=0.3; on the production retry path (t=0.9, top_p 1.0, presence_penalty 1.8) S07 is 0.45/M [0.04; 1.17] — not exactly zero, with no observed event. This one is itself a lower bound: with top_p 1.0, ties at the top-k boundary keep more than 20 candidates at ~4 % of positions.
• —Zero observed CJK tokens for `S07` across three freshly started server instances, 15 seeds × 3 sampling profiles, a 3-way concurrent batch, and the 150 synthetic documents under the production sampler; the base produced 65 CJK tokens on the same seeds. A post-closure recount of every record — including tool-call turns and runaway generations, which the risk calculator skips — keeps S07 at zero over ~1.09 M distinct positions (Poisson 95 % upper bound < 2.8/M) and adds one 6 593-CJK-token runaway for the base on the retry profile.
• —The patch does not cure degenerate generations. On the retry profile (t=0.9, top_p 1.0) 2 of 15 runs ran away into 35–65 k tokens of word salad — on the base (with CJK) and on S07 (without). Keep your loop guard.

4. What was not measured

• —Other models. One checkpoint, one quantisation (FP8), one engine (vLLM). Nothing here transfers to another model without re-measuring; the mask depends on the vocabulary.
• —Japanese and Korean. The Chinese probe is the only CJK capability instrument. Kanji and hanja share the targeted Unicode ranges, so the loss is expected to be the same — but it was not measured.
• —Other languages than Hungarian on the benefit side. The mechanism is language-agnostic (it targets a token class, not a language), the evidence is Hungarian.
• —Long context beyond ~15 k tokens. The Hungarian agent traces are 13–19 k tokens; the synthetic documents are shorter.
• —Untruncated samplers (see §2).
• —The width of the window. S = 0.7 is the smallest dose measured with all-zero risk and intact Chinese; S = 0.8 was zero on the production path (t ≤ 0.8) but was not re-measured on the actual support. Nothing between 0.6 and 0.7 was measured.

5. Measurement caveats you should know before quoting numbers

• —Server-instance non-determinism. On this hardware/engine, greedy output is bit-stable within one server process but differs between freshly started processes on knife-edge items (2–5 of 150, in at least five discrete "modes" over 16 starts). Because the patch provably does not change Hungarian greedy output, any difference in the 150-item column identifies the instance mode, not the arm. VLLM_BATCH_INVARIANT=1 with --attention-backend TRITON_ATTN was verified to be reproducible across instances and batch sizes (~12 % slower); the published numbers were not measured in that mode.
• —The internal extraction benchmarks (customer invoices and contracts) showed no regression on any arm; they are confirmatory only and cannot be published.
• —Three customer traces are the source of the Hungarian agent-trace risk numbers; they are not public. Arms served in the same instance mode follow identical sampled trajectories on them (shared server RNG stream), so the zeros of the scale arms in the table are not independent samples, and a bootstrap CI of [0; 0] is a structural statement about the visited positions, not a statistical bound — the statistical bound is the Poisson one above. The 150 synthetic documents and the Chinese probe are.

6. Reproduce it

reproduction/ contains everything needed to rebuild this checkpoint from the base and to re-run the two publishable instruments:

The 150-item Hungarian trap corpus, its harness and the 36-item Chinese probe live in the public measurement repository (link below), together with the per-arm outputs on the synthetic corpus.

Rebuild and verify:

python3 reproduction/celtokenek.py --tokenizer <base> --out celtokenek.json
python3 reproduction/patch_lm_head.py --snapshot <base> --celtokenek celtokenek.json \
    --out ./cjk-damped-S07 --kar S07 --forma szorzas --skala 0.7 --maszk nyers
python3 reproduction/ellenoriz_kar.py --alap <base> --kar ./cjk-damped-S07 --celtokenek celtokenek.json
sha256sum ./cjk-damped-S07/outside.safetensors   # 3cbfe634c523cc2e622679847bdc78412073dc38b738a661451995f2c0a0bfc5

7. Attribution and licence

• —Base model: Qwen/Qwen3.6-35B-A3B-FP8, Apache-2.0. This checkpoint is released under the same Apache-2.0 licence; the modification is the one described in §1 and nothing else.
• —Target-class definition: dnotitia/smoothie-qwen (the Unicode-range mask, used unchanged as the "raw" mask).
• —Closest prior work and the direction-based form + language-specific mask that the A* and *F arms test: ThakiCloud/Qwen3.8-27B-ko-cjk-suppressed. Their model card's warning applies here in reverse: do not reuse this mask on another language as-is — what Korean must keep, Hungarian can drop, and vice versa.
• —A naming note for search: "smooth" on the Hub usually means SmoothQuant, a quantisation technique. This is unrelated; hence cjk-damped.

8. Study and measurement package

• —Study (Hungarian, pre-registered protocol, all hypotheses and negative results): https://docai.hu/kutatas/cjk-csillapitas
• —Public measurement package (code, synthetic corpus, per-arm outputs, result sheet): https://github.com/k3net/docai-evals/tree/master/experiments/2026-09-18-cjk-damping-dose-response-gb10

@misc{kiss2026cjkdamped,
  title  = {How much may be taken from a token class? A dose--response study of CJK damping in the output head of Qwen3.6-35B-A3B-FP8},
  author = {Kiss, D{\'a}niel},
  year   = {2026},
  note   = {K3Net Kft. / DocAI. Model: Qwen3.6-35B-A3B-FP8-cjk-damped-S07. Pre-registered protocol 2026-09-15, measurements 2026-09-18--20.},
  url    = {https://docai.hu/kutatas/cjk-csillapitas}
}