ManniX-ITA/Qwen3.8-27B-Omnimerge-v6

Qwen3.8-27B-Omnimerge-v6

Task-arithmetic merge of three Qwen3.6 fine-tunes onto the Qwen3.8-27B base — the same sources, weights and method as `ManniX-ITA/Qwen3.6-27B-Omnimerge-v4`, moved to the newer base generation. No fine-tuning and no distillation of its own: weight-space arithmetic over published checkpoints only.

Named v6 because v5 is taken by Qwen3.6-27B-Omnimerge-v5-mlp-skip.

Headline: on a five-bench sampled cohort run head-to-head against v4 on the same binary and sampler, the one result outside the measurement band is LiveCodeBench 0.883 vs 0.818 (+6.5 pp, 68/77 vs 63/77). Everything else ties or sits inside noise; GPQA is 2 pp lower. v6 also thinks substantially longer for it — see Thinking profile.

Sources

Method: omnimerge_v2 (DARE-TIES base + OBIM-lite + DAREx q + EMR election). Density 0.53, DAREx q 0.75, seed 42. mlp.gate_proj / up_proj / down_proj and mtp. are passed through from the base.

The task base is load-bearing

Every source was fine-tuned from Qwen3.6, so each delta must be taken against 3.6 and then applied to 3.8. Merging without --task-base computes delta = source − 3.8, which embeds the inverse of the 3.6→3.8 generational upgrade and drags the result backwards — and on the 7 embedding rows 3.8 adds, it would compute (3.6 padding) − (3.8 audio embedding) and corrupt them.

Proof the transplant landed, from verify_merge_artifact.py on a sampled q_proj:

||out − Qwen3.8|| =  1.09      <- output sits on the 3.8 base
||out − Qwen3.6|| = 67.78

Verified, not assumed

Vocabulary. vocab.json and merges.txt are sha256-identical between Qwen3.6-27B and Qwen3.8-27B. 3.8 adds 7 tokens purely additively at ids 248070–248076 (audio/TTS); no id is reassigned or dropped. All three sources match the 3.8 vocab exactly — their merges differ only in serialisation ("Ġ Ġ" vs ["Ġ","Ġ"]).

Vision tower. All 333 model.visual.* tensors (167 weights + 166 non-weights) are bit-identical between the merge and the 3.8 base, so the stock Qwen3.8 projector is the correct mmproj.

MTP head is deliberately not merged. All three sources carry the 15 mtp.* tensors, so the default would apply a 3.6-trained head's delta to 3.8's head. MTP affects draft-acceptance rate only — it costs decode speed, silently. 3.8's head is preserved verbatim and appears in the GGUF as blk.64.* (blocks 0..64, 866 tensors). MERGE_MTP=1 merges it instead.

Benchmark Results

Sampled cohort (recommended: temperature 0.6 / top-p 0.95 / top-k 20, do_sample=true), Q6_K, llama.cpp, identical benches and sampler across every cell.

Comparable to each other only — do not pool with greedy-decode results from any other card. The greedy GPQA table further down is a separate cohort: its rows must never be merged into these.

1. Calibration change — previous imatrix vs AtomicChat

The published ladder had been calibrated on a 128-chunk in-repo corpus, and six K-tiers (Q6KL, Q6K, Q5KL, Q5KM, Q4KL, Q4KM) carried **no imatrix at all** — an exclusion policy measured on a different model family that should never have applied here. (Q80 also carries none, but that is correct: it is imatrix-free by rule.) Every _K/IQ tier is being rebuilt on AtomicChat/calib-corpora builds/qwen3.8-27b at the full 9,703 chunks, ~76× the previous basis. The Q6_K above is the recalibrated file; the remaining tiers are re-uploading and this note will be removed when the ladder is complete.

Read these four deltas as a wash, not a win or a loss. The two negatives (IFEval −2.00 pp = 2 questions of 100; HumanEval −0.60 pp = 1 of 164) and the one positive (MultiPL-E +0.60 pp) are all inside single-draw sampled noise, and LiveCodeBench is unchanged to four decimals. The case for recalibrating is not a score gain — it is that seven tiers were shipping uncalibrated, and that an omitted imatrix is the asymmetric error: at worst it costs ~1 pp where it is neutral, while omitting one where it matters is catastrophic.

2. Omnimerge-v6 vs Omnimerge-v4

This table is kept whole and unchanged on the previous-imatrix Q6_K column, because it is a controlled v6-vs-v4 pair. Swapping some cells to the recalibrated quant while others stayed would make the row set mixed-basis, which is worse than slightly stale; the calibration change is measured separately in table 1.

Only LiveCodeBench is outside the measurement band (+6.5 pp = 5 more solved of 77). The ±1 pp cells are not differences — a same-basis MultiPL-E-100 repeat on this rig moved 1.0 pp from batch scheduling alone. GPQA's −2 pp is 4 questions of 198: suggestive, not established. The mean is carried almost entirely by LiveCodeBench.

3. GPQA — greedy only, paired

For the calibration question, GPQA is reported only from a greedy paired run, and is deliberately absent from table 1. Its sampled cell there was a single noisy draw reading −5.05 pp for the AtomicChat quant; a paired greedy rerun of that identical pair read +2.02 pp — the opposite sign, so the sampled number would have misrepresented the change. (GPQA remains in table 2, where both columns sit on the same sampled basis and the comparison is v6-vs-v4 rather than imatrix-vs-imatrix.) The greedy pair is the trustworthy measurement: both arms ran the same morning, concurrently, on the same host and binary, scored per-question on the same 198 items.

              ac_correct  ac_wrong          delta(ac-prev)  = +2.02 pp
prev_correct     143         11             McNemar exact p = 0.5572
prev_wrong        15         29             95% CI on delta = [-3.02, +7.06] pp
                                            discordant      = 26/198 = 13.1%

p = 0.5572: the two quants are statistically indistinguishable on GPQA. This is not a win. Its value is the interval — it excludes −5 pp, so the degradation the sampled cell suggested is ruled out at 95%. Degenerate outputs (empty or [invalid]) were 5 for the previous imatrix vs 2 for AtomicChat, and response lengths are comparable (p50 1142 vs 1187 chars), so nothing is hiding under the equal means.

4. Tool-calling — tool-eval-bench hardmode (88 scenarios, 176 pts)

v6 is first of ten in this cohort, and the safety result is the one worth reading.

[image]

v6 scores +10.2 pts over v4 and +5.6 over its own Qwen3.8 base, with non-overlapping CIs against v4.

The safety number is not a rounding difference. Every other model in the cohort — including both bases — fails TC-60 (Cross-Turn Sleeper Injection) on 5 of 5 seeds. v6 never fails it. Its 3 failures across five runs are 2× TC-58 (Fake System Message in File) and 1× TC-33 (Hallucination Resistance); it has no TC-31 or TC-34 failure at all, where v4 fails both on every seed. Safety & Boundaries 23.2/26 (89.2%) vs v4's 18.4/26 (70.8%).

Category profile (mean over 5 seeds): perfect on Tool Selection, Parameter Precision, Restraint & Refusal, Localization, Instruction Following, Toolset Scale and Creative Composition. Weakest at Autonomous Planning 3.4/6 (56.7%) — below v4's 4.0/6 — and Context & State 14.8/20 (74.0%). Hard Mode 33.0/38 (86.8%) leads the cohort, but only narrowly — the Qwen3.8 base is 32.8/38 (86.3%). The wide gap is Safety & Boundaries: 89.2% here vs 78.5% for the next-best model.

<details> <summary><b>Basis — read before comparing these numbers to anything</b></summary>

• —Scorer: `tool-eval-bench` v2.6.0 (the pip/uv-installed package, verified via tool_eval_bench.__file__, not a git checkout). An earlier note in the runner claimed cf54b4b (v2.6.0-45); that is wrong and has been corrected — no cell ever ran it. All 50 cells in this cohort ran the same v2.6.0, so the cohort is internally consistent.
• —v2.6.0 carries a known scorer crash on TC-62. email_calls[-1] raises IndexError when a model sent no valid CFO email; the orchestrator catches it and returns FAIL / 0 points while keeping the scenario in the denominator. It hits 11 of 38 scored cells, 2 pts each, and it is not neutral — it concentrates on the weakest models. Later harness commits credit that behaviour instead, so the fixed scorer would raise affected scores unevenly. v6 has no affected cell; v4 has none either, so the v6-vs-v4 gap above is not exposed to it.
• —5 paired seeds [42–46], 64k context, context-pressure 0.25, max 8 turns, 120 s timeout, thinking enabled, sampler temp 0.6 / top-p 0.95 / top-k 20 (not greedy).
• —Served on llama.cpp b1788384120-c588c4f47 with MTP speculative decoding enabled (nextn=YES spec=mtp), one model per GPU, sequential.
• —Quant tiers are not uniform across the cohort (Q4KM here, IQ4XS for the Ornith and 35B-A3B rows, UD-Q4K_M for the Qwen3.8 base). Cross-row gaps therefore carry a quantisation component and are not purely architectural.
• —Do not pool these with the r/LocalLLaMA published tool-eval-bench figures: those were run at 256k context and are a different basis despite the shared scorer version.

</details>

Quantizations

GGUF (llama.cpp / ollama / text-generation-webui) — 19 tiers with the MTP head retained, plus the F16 vision projector: `ManniX-ITA/Qwen3.8-27B-Omnimerge-v6-MTP-GGUF`

ollama — `mannix/omnimerge-v6` (:latest = Q4KM). Tags carry the same serving identity as the official qwen3.8:27b (RENDERER qwen3.8, PARSER qwen3.5, vendor sampling defaults), so they need ollama >= 0.32.12.

ollama run mannix/omnimerge-v6

Reproducing the merge

python omnimergekit.py \
    --base      Qwen/Qwen3.8-27B \
    --task-base Qwen/Qwen3.6-27B \
    --source    rico03/Qwen3.6-27B-Claude-Opus-Reasoning-Distilled \
    --source    ValiantLabs/Qwen3.6-27B-Esper3.1 \
    --source    <kai-os LoRA applied to Qwen3.6-27B> \
    --weights 0.40,0.35,0.25 \
    --method omnimerge_v2 \
    --density 0.53 --darex-q 0.75 --seed 42 \
    --skip-patterns 'mlp.gate_proj,mlp.up_proj,mlp.down_proj,mtp.' \
    --output Qwen3.8-27B-Omnimerge-v6

The kai-os source is a LoRA; it is applied to Qwen3.6-27B first (scripts/apply_lora_to_safetensors.py) and the resulting anchor is merged as a full model. Tokenizer, chat template and preprocessor configs are copied from the 3.8 base after the merge.

Merge engine: mann1x/omnimergekit.

Caveats

• —Research checkpoint. One eval cohort, one seed, one quantization tier (Q6_K). The only delta established beyond the noise band is LiveCodeBench.
• —Verbose. Budget more thinking headroom than you would for v4; six GPQA questions scored zero on budget exhaustion alone.
• —Benchmarks were measured on Q6_K. Lower tiers are not separately evaluated.
• —Sampled-cohort numbers throughout — not comparable to greedy tables elsewhere.

Acknowledgements

Qwen team for the Qwen3.8 base and the vision/MTP components; rico03, ValiantLabs and kai-os for the source fine-tunes.