windowsxp811203/qwen3.8-conv1d-sigscalesync

The conv1d sigma outliers in Qwen3.8-27B are real. The quantization story attached to them is not — and the rescale variant I tested cost perplexity while fixing nothing measurable.

Scope revision (2026-08-24). An earlier version of this README said the rescale was applied "exactly as published" and that the mechanism "cannot work". Both were too strong. What I tested is a 7-tensor median-normalization variant (σ > 1.5× median → α = median/σ); the intervention actually published for Qwen3.8-27B targets 8 layers (including layer 61) with author-specific per-layer α values that do not equal median/σ under my σ measurements. And llama.cpp's F32 exclusion rules out direct quantization of the conv1d weights as a mechanism in these files — it does not rule out effects already present in unquantized weights, which is exactly what the pre-registered A/B below was designed to measure. The data tables are unchanged; the claims around them are narrowed.

In August 2026 a patch called Sig-ScaleSync circulated for Qwen3.5/3.6/3.8-class models. Its claim: a handful of linear_attn.conv1d tensors carry an abnormally wide weight distribution, this gets worse under quantization, and the result is model collapse in long multi-turn sessions — looping, incoherence, tool calls breaking around 100K tokens. The remedy: rescale any conv1d tensor whose sigma exceeds 1.5× the median by alpha = median / sigma.

This repository is what happened when I checked it against my own weights instead of applying it. Everything here is a script you can run and the output it produced.

1. The premise is correct

check_conv1d.py computes sigma for all 48 linear_attn.conv1d.weight tensors in Qwen3.8-27B. Median sigma is 0.04309, and seven blocks exceed 1.5× median — layers 52, 53, 56, 57, 58, 60 and 62, ranging from 1.65× to 1.92×. Layers 54 and 61 sit just under the line at 1.46× and 1.49×. This closely matches the list published for Qwen3.6-27B, and it is inherited from the base model: abliteration never touches conv1d.

Full table in `CONV1D_SIGMA.txt`.

python check_conv1d.py /path/to/Qwen3.8-27B      # needs torch + safetensors

So the pattern the patch is named after exists. That is where agreement ends.

2. Direct quantization of these tensors cannot be the mechanism — llama.cpp never quantizes them

The patch's causal story requires quantization noise to interact with the conv1d weights. It cannot, because llama.cpp excludes them from quantization outright. In llama-quant.cpp:

// do not quantize Mamba/Kimi's small conv1d weights
quantize &= name.find("ssm_conv1d") == std::string::npos;

and gguf-py/gguf/tensor_mapping.py maps this architecture's linear_attn.conv1d onto exactly that ssm_conv1d name.

Rather than trust the source, gguf_conv1d_dtypes.py reads the tensor-info table straight out of the shipped GGUF files. Across BF16, Q4KM, Q5KM and IQ4XS, **all 48 conv1d tensors are stored as F32** while the rest of the file is Q4K/Q5K/Q6K/IQ4XS. Output in [`CONV1DDTYPES.txt`](CONV1D_DTYPES.txt).

python gguf_conv1d_dtypes.py model.gguf          # header-only, no dependencies

The allegedly quantization-damaged tensors are already at full precision in every file I ship.

3. Applying the patch costs perplexity and buys nothing measurable

I built a median-normalization variant of the patch (σ > 1.5× median → α = median/σ, which selects 7 tensors; the published Qwen3.8 intervention targets 8 layers with different per-layer α) — byte-verified as the only change — and compared both at Q4KM on held-out wikitext:

The penalty is real, and more tellingly it is flat. A defect that damages long context should show a widening gap as context grows. It doesn't move from 4K to 256K — the signature of slightly detuned trained weights, not of a masked instability.

Needle-in-a-haystack retrieval was clean for both builds out to 823,868 real prompt tokens. A 90-turn dialogue with KV reuse (132K tokens, recall probes every 6th turn) completed 90/90 with 15/15 probes correct, minimum distinct-5-gram ratio 1.000, on both builds. Three different methods, no reproduction of the reported failure.

4. The steelman, and the experiment that was never triggered

There is a version of the concern worth taking seriously, and it is not about quantization. In this architecture Q and K pass through normalization after the convolution, so conv scale there largely cancels — but V does not, and the recurrent state S is not re-normalized between tokens. Oversized V-writes into near-1-decay heads saturating the state is a genuine failure class for linear RNNs.

That is a measurable prediction, so I built a pre-registered A/B to measure it, published the design before running it, and offered to run it on my hardware:

• —BF16 unmodified vs BF16 Sig-ScaleSync, same engine commit, same host, speculative decoding off
• —the sampler the reporter specified (temp 1.15, top-p 1.0, top-k 0, min-p 0.06, rep-pen 1.05) plus a control
• —10 paired seeds × 141 scripted user turns (one setup + 140 test turns, ~200K tokens): persona role-play, nested checkable reasoning, high-entropy symbol blocks, recall probes spread across the context
• —instrumented at the mechanism: per-chunk Frobenius norm of the DeltaNet state S for the 7 outlier layers against mid-stack controls, by teacher-forced replay of every transcript. If the theory holds, state norms diverge before the text degrades.
• —collapse criteria fixed in advance, including character-level n-gram checks so CJK loops count, and infrastructure failures separated from model failures so a 503 can never be scored as a collapse

The design is in `PREREG.md`; the harness is ab_gen_turns.py, ab_driver.py, ab_replay_norms.py, ab_server.sh, ab_orchestrate.sh, with the frozen scripted-turns file turns.jsonl (141 records: one setup + 140 test turns; sha256 begins 61bf02ca — PREREG.md's "140" counts the test turns) and build hashes in gguf_hashes.txt.

The full 40-session matrix was never run. It was conditioned on receiving a session or script that reproduces the failure, and none was ever provided. The harness passed its smoke test and is published here unrun, so that anyone who does have a reproduction can execute a design that was fixed before anybody saw the outcome. Publishing it unrun is the point: a test you can only run after you know the answer is not evidence.

What I am not claiming

That the patch is harmful to every model — I measured this model, at these quant levels, on these tasks. That no long-context failure exists in Qwen3.8-27B — I could not reproduce one across three methods, which is not the same as proving absence. That sigma outliers are meaningless — they are observably there, and the steelman above may yet be right about V-path saturation under sampling regimes I did not reach.

What I am claiming is narrower and checkable: quantization does not touch these tensors in these files, so direct quantization of them cannot be the mechanism there; and the median-normalization variant I tested worsened perplexity at every context length while improving no other measured outcome — completion, recall, repetition and needle retrieval were identical between the two builds. Whether the published 8-layer per-layer-α intervention behaves differently is untested here; the harness in this repo is how you'd find out.