Modelpublic

gbuzhf/Ornith-1.5-35B-A3B-TIEL-Calibrated-MTPv2-ICE-GGUF

sourceHugging Facemitupdated 16d agoView on Hugging Face

9likes5.7kdownloads

Model Card

Ornith-1.5-35B-A3B — TIEL-Calibrated MTPv2 ICE tiers

Four GGUF tiers of the original ornith-ai/Ornith-1.5-35B-A3B, with ornith-ai's trained MTPv2 head embedded, calibrated on Tiel's importance matrix, carrying the Qwen-Sharp v22.4.1 chat template.

Which one should I download?

tier	size	mean KLD	vs the Unsloth Dynamic ladder
`23G-ICE`	22.84 GB	0.0325	Beats `UD-Q4_K_XL` on both axes — 14.5 % closer to bf16 and 0.37 GB smaller. Best value here.
`25G-ICE`	24.85 GB	0.0284	The best file below 25 GB — nothing smaller is closer to bf16. Also beats `APEX-I-Balanced` (26.28 GB / 0.0345) by 12 % while being 1.4 GB smaller. Above it, `UD-Q5_K_S` at 25.83 GB is stronger.
`21G-ICE`	20.85 GB	0.0389	Fills a 4.5 GB hole in the UD ladder. Essentially `UD-Q4_K_XL` quality (0.0380) at 2.36 GB less.
`19G-ICE`	18.82 GB	0.0601	17 % closer to bf16 than `UD-IQ4_XS` (0.0723) for +0.14 GB.

Short version: ICE wins in the 19–23 GB band and loses above 25 GB — see Method.

Measurements

KL divergence against the BF16 master this repo was built from. WikiText-2 raw test, 64 chunks, n_ctx 2048, one binary and one reference for all four files. Mean PPL(base) = 7.494953 ± 0.079236.

Sorted best → worst by overall (BF16 = 100), the same composite used on the other Ornith-1.5 cards: 0.70/(1+meanKLD) + 0.30*sameTop1.

tier	size	mean KLD	99% KLD	99.9% KLD	PPL ratio	same top-1	active bpw	file bpw	overall
`25G-ICE`	24.84 GB	0.0284	0.286	1.120	0.9851	93.44%	7.686	5.597	96.1
`23G-ICE`	22.83 GB	0.0325	0.335	1.177	0.9900	92.85%	7.523	5.143	95.7
`21G-ICE`	20.84 GB	0.0389	0.419	1.584	1.0001	92.24%	7.357	4.695	95.1
`19G-ICE`	18.82 GB	0.0601	0.616	2.387	1.0013	90.33%	7.192	4.240	93.1

Active bpw weights each tensor by how often it actually runs — routed experts at k/E — so it says where the bits went in the forward pass rather than on disk. It explains a design; it does not rank one. Ranking is on measured KLD.

Method

Every GGUF quantizer — llama.cpp's own mixes, Unsloth Dynamic, APEX — minimises the same thing for every tensor: importance-weighted error of that tensor's output, for the current token. That is correct for a tensor whose error dies with the token, and wrong for the ones whose error does not.

ICE sorts tensors by how far an error travels, then pays accordingly:

class	what it is here	why	type
discrete	the MoE router	an error flips an argmax, so a different expert runs. Not a graded loss — a categorical one.	F32
recurrent	SSM decay / timestep terms	the error enters a carried state and compounds along the sequence	F32
cached	`attn_k`, `attn_v`	written to the KV cache once, re-read by every later token, never re-decided	F16
instant	everything else, incl. all 256 experts	the error affects this token only	the dial

The first three classes are 0.14 % of the model — 47 M parameters, 0.15 GB. Freezing them outright is a line item, not a trade-off. The recovered budget plus the whole remaining budget goes to the expert bank, uniformly across gate/up/down, with the higher type placed shallow-first.

Where the budget goes furthest. All four tiers here are Pareto-optimal against a twelve-tier comparison of the Unsloth Dynamic and APEX ladders on this model, measured on one harness against one reference: nothing published is both smaller and closer to bf16 than any of them.

The advantage is largest in the 19–23 GB band and narrows as the budget rises: above ~25 GB the expert term saturates and what remains is the dense path, which is the regime UD's "pin the dense path high" policy is built for. That is why UD-Q5_K_S and UD-Q6_K remain the strongest files on the board, and why this ladder stops at 25 GB rather than chasing them.

Full report, including seven negative results and the retraction of a rule this work itself derived and shipped: [gbuzhf/ICE-quantization](https://huggingface.co/gbuzhf/ICE-quantization)

Files and use

Ornith-1.5-35B-A3B-TIEL_Calibrated-MTPv2-{19,21,23,25}G-ICE.gguf

Each is a single file carrying the MTPv2 head (block_count=41, 753 tensors) and the Sharp template. For self-speculative decoding pass --spec-type draft-mtp; the head is embedded, no sidecar model is needed