fetchingcat/occamy-ornith-35b-a3b-mtp-gguf

Occamy × Ornith — 35B-A3B MTP (GGUF)

A weight-space merge of two independent full-parameter finetunes of Qwen/Qwen3.6-35B-A3B:

Q4KM with imatrix, 35.5B total / ~3B active. Multi-token prediction (MTP) speculative decoding and vision are both intact.

Neither parent ships an MoE merge with working MTP, so the main point of interest here is that the speculative-decoding head survives weight-space merging — draft acceptance is actually higher than either parent measured under identical settings (details below).

At a glance

Statistically indistinguishable from Occamy on every capability benchmark, not significantly worse than Ornith on any of them, and faster than both. The single capability regression is perplexity against Occamy.

Files

The merged BF16 safetensors are not currently uploaded — the config and scripts above reproduce the model exactly from the public parents. They may be added later.

Merge recipe

merge_method: task_arithmetic
base_model: Qwen/Qwen3.6-35B-A3B

models:
  - model: ./occamy-fused            # normalized checkpoints, see "Reproducing"
    parameters:
      weight: 1.0
  - model: ./ornith-fused
    parameters:
      weight: 0.3

parameters:
  normalize: false

dtype: bfloat16

normalize: false is task_arithmetic's default but is set explicitly: with it on, the summed delta is divided by sum(weights) = 1.3 and both contributions shrink.

The per-model `weight` is not "share of the result." Ornith's task vector is 3.0–3.4× larger than Occamy's on every component, so 0.5/0.5 would have given Ornith ~3.4× the actual movement. 1.0/0.3 keeps Occamy's full task vector and adds an equal-magnitude contribution from Ornith. Measured realized coefficients (⟨merged−base, δᵢ⟩ / ‖δᵢ‖²) were +1.01…+1.05 for Occamy and +0.305…+0.311 for Ornith, matching the design.

task_arithmetic rather than TIES/DARE/SCE: the two task vectors are near-orthogonal (cosine 0.01–0.11), so there is essentially no interference for sign-election to resolve, and sign election instead amplifies whichever delta is larger. Under SCE the measured effective coefficients were ~+0.04 Occamy / ~+0.80 Ornith — i.e. the result would have been close to plain Ornith.

The 19 mtp.* tensors are not merged; they are taken verbatim from Ornith-1.5. mergekit drops them because transformers does not implement MTP for this architecture, and averaging a trained MTP head with one that was never trained against its backbone is not obviously meaningful.

Usage

Ollama

ollama create occamy-ornith:35b-a3b-mtp -f Modelfile

FROM ./occamy-ornith-35b-a3b-mtp-Q4_K_M.gguf
FROM ./occamy-ornith-mmproj-F16.gguf

TEMPLATE {{ .Prompt }}

PARAMETER draft_num_predict 2
PARAMETER num_ctx 262144

PARAMETER num_predict 32768
PARAMETER repeat_last_n 4096
PARAMETER presence_penalty 1.5
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

PARAMETER temperature 1
PARAMETER top_k 20
PARAMETER top_p 0.95

The loop guards are not optional. llama.cpp applies presence_penalty only over the last repeat_last_n tokens, and Ollama defaults that to 64 — far too narrow to catch multi-paragraph repetition, which makes the penalty inert. 4096 is required. Occamy-1.0 looped badly without this and the behaviour carries over.

Keep TEMPLATE {{ .Prompt }} as a passthrough so llama-server applies the GGUF's own Jinja template. Substituting a hand-written Go template that emits an empty <think></think> will silently disable reasoning.

MTP requires flash attention — run the Ollama container with OLLAMA_FLASH_ATTENTION=true.

llama.cpp

llama-server -m occamy-ornith-35b-a3b-mtp-Q4_K_M.gguf \
  --mmproj occamy-ornith-mmproj-F16.gguf \
  --spec-type draft-mtp --spec-draft-n-max 2 --flash-attn on \
  -c 262144

Measured results

All figures below are this Q4KM build measured on the same machine (DGX Spark / GB10). Parents were measured from their own Q4KM GGUFs under identical templates and sampling parameters — note that the stock ornith1.5 Ollama tag ships a different template and stop set, which skews comparisons if not normalized.

Speculative decoding and throughput

3 prompts, 512-token cap, temperature 0, seed 42, fresh runner per model, each model measured twice in opposite orders.

Draft acceptance is higher than either parent, including Ornith's own trained MTP head running on Ornith's own backbone. A plausible explanation is that the merge sits closer to base than Ornith does, and the MTP heads were trained against base-like distributions — but this is an inference, not something we tested.

Capability

Because every model sees the same items, the informative test is paired (McNemar over discordant items), not the headline percentages:

No statistically significant difference from either parent on either benchmark. HellaSwag and Winogrande are likewise within overlapping confidence intervals of Occamy.

Against Ornith the picture is one-sided: Winogrande is a clear win (72.22 [70.96–73.48] vs 69.61 [68.32–70.91], non-overlapping intervals), as are perplexity, throughput and draft acceptance — and nothing measured comes out significantly worse.

Perplexity against Occamy is the single capability regression (7.21 vs 6.66, ~8% worse).

Where the parents actually differ

Aggregate scores hide the only axis on which the two parents separate. Broken out by BFCL category:

simple is the only category where the parents differ significantly (ornith-1.5 362 vs occamy-1.0 346, McNemar p = 0.0052). On every other category they are statistically indistinguishable — and a benchmark on which the parents tie carries no information about what the merge inherited, whatever the merge scores on it.

Is it just Occamy?

A merge that tracks one parent this closely invites the question of whether it is simply a re-badge of it. Counting how often two models emit identical tool calls across the 1,240 BFCL cases — function name and arguments, ignoring the generated call ids:

The merge is closest to Occamy, consistent with the realized coefficients — but only by 2.1 points over how much the two parents already resemble each other. A re-badged Occamy would agree with Occamy at close to 100%.

Dose-response

To test whether Ornith's simple advantage transfers with merge weight, a second merge was built identically except with Ornith at weight: 0.6 instead of 0.3:

The effect moves in the expected direction and roughly with dose: at w=0.3 the model is still significantly below Ornith (p = 0.019), while at w=0.6 it is not (p = 0.134). This is weak evidence — one category of one benchmark — but it is causal rather than correlational, since the merge coefficient is the only thing varied.

w=0.3 is released rather than w=0.6 because the larger Ornith contribution measured worse on decode throughput, draft acceptance and calibration perplexity, in exchange for a gain confined to that single category. Those comparisons were run in a separate session from the throughput table above and are deliberately not quoted here as if they were the same measurement.

Evaluation scope

A snapshot of what has been measured so far. This section will be updated as further evaluation is done.

Measured: draft acceptance and decode throughput; BFCL v3 (non-live AST, 1,240 cases); pairwise behavioural agreement on emitted tool calls; HumanEval+; HellaSwag; Winogrande; wikitext-2 perplexity; vision projector integrity; repetition/loop checks; numerical integrity of the merged weights.

Not yet measured:

• —Repo-scale coding and multi-turn agentic loops. BFCL v3 above covers single-turn tool-call construction, not stateful agent sequences or editing an existing codebase — which is what Ornith is built for (it reports SWE-bench Verified 79). Aider polyglot was attempted and the runs were discarded for harness faults rather than model behaviour, so nothing is reported here.
• —Instruction-following. Occamy reports IFEval 91.53; not independently verified here.
• —Reasoning. GPQA Diamond was attempted and set aside: these models are verbose enough that the score became a function of whether they finished inside the token budget rather than whether they reasoned correctly, which is not a measurement worth publishing.

How to read the numbers:

• —HumanEval+ is near ceiling here — with all three models at 91–93% only 13–15 problems discriminate between them, so it confirms no regression rather than ranking coding ability. HumanEval also dates to 2021 and is well represented in training data; EvalPlus's additional tests address weak testing, not contamination.
• —On BFCL the three models sit within 1.2 points of each other overall, and four of the five categories show no significant parent difference at all. Only simple discriminates, which is why the dose-response check above is reported on that category alone rather than on the aggregate.
• —The capability results are consistent with the merge tracking Occamy closely with a weighted Ornith contribution. On decode throughput and MTP draft acceptance it exceeds both parents outright; on the capability benchmarks it matches Occamy and is never significantly worse than Ornith. No capability benchmark showed it exceeding both parents.

Reproducing

The parents store experts in inverted layouts — Occamy stores the main body unfused (one nn.Linear per expert, 30,720 tensors) while Ornith stores it fused 3-D and inverts this for its MTP block. mergekit keeps only tensor names present in all models, so merging them directly silently drops every expert weight and yields a "successful" merge missing ~95% of its parameters.

qwen36-moe-fuse.py normalizes both to the fused 3-D layout first, following llama.cpp's own split convention so the GGUF round-trip is exact:

gate_up_proj [n_exp, 2*n_ff, n_embd] = cat(stack(gate), stack(up), dim=1)
down_proj    [n_exp, n_embd,  n_ff]  = stack(down)

After normalization all three checkpoints present identical 1,045-tensor manifests with zero shape mismatches.

python qwen36-moe-fuse.py <occamy_dir> occamy-fused
python qwen36-moe-fuse.py <ornith_dir> ornith-fused
mergekit-yaml merge-occamy-ornith.yaml merged --copy-tokenizer
python graft-mtp.py ornith-fused merged merged-mtp
# mergekit does not copy preprocessor_config.json; copy it from the base model
# or convert_hf_to_gguf.py --mmproj fails with KeyError: 'image_mean'
python convert_hf_to_gguf.py merged-mtp --outtype bf16 --outfile merged-BF16.gguf
llama-imatrix -m merged-BF16.gguf -f calibration_datav3.txt -o imatrix.gguf --chunks 129
llama-quantize --imatrix imatrix.gguf merged-BF16.gguf out-Q4_K_M.gguf Q4_K_M

Verification checkpoints: merge plans 1,026 tensors (the 19 missing are all mtp.*); the merged model has 35,107,181,936 parameters, exactly one MTP block short of Ornith's total; the GGUF has 753 tensors with block_count 41 and nextn_predict_layers 1; the vision projector is bit-identical to the base tower (0 of 334 tensors differ), since neither parent modified it.

imatrix used bartowski's calibration_datav3.txt (129 chunks, PPL 7.4508), which gave 99.2–99.6% per-expert coverage on the 256-expert MoE.

License and attribution

Released under Apache-2.0, inheriting from Qwen3.6-35B-A3B (Apache-2.0) and occamy-1.0 (Apache-2.0), and incorporating weights from Ornith-1.5-35B-A3B (MIT). LICENSE-ornith reproduces the MIT terms in full and attributes copyright to ornith-ai; note that the upstream Ornith repository declares MIT in its metadata but ships no LICENSE file, so there is no upstream copyright line to reproduce verbatim. This is a modified derivative work of all three; the modifications are the task-arithmetic merge, MTP graft, and quantization described above.

Full credit for the underlying models belongs to the Qwen team, Accio Lab, and ornith-ai. This repository contributes only the merge, packaging, and the measurements above.