Tribbler/ornith-1.0-apex

Ornith-1.0-35B — APEX GGUF

APEX (Adaptive Precision for EXpert Models) quantizations of Ornith-1.0-35B, an open-source coding MoE model by DeepReinforce (MIT license, based on Qwen 3.5 architecture).

These quants were produced using the apex-quant toolchain. APEX is a MoE-aware mixed-precision quantization strategy that classifies tensors by role (routed expert, shared expert, attention) and applies a layer-wise precision gradient — edge layers get higher precision, middle layers more aggressive compression.

Files

Each profile comes in two variants:

• —Base — the quantized model standalone
• —-MTP — includes the bundled MTP (multi-token prediction) head, quantized to Q8_0 (near-lossless), for self-speculative decoding via --spec-type draft-mtp. Requires a recent llama.cpp build with MTP support.

I-variants were calibrated with a diverse importance matrix (chat, code, reasoning, tool-calling, multilingual) for improved downstream accuracy.

Profile Precision Breakdown

APEX applies a layer-wise precision gradient to MoE expert weights. I-variants additionally use a diverse imatrix (chat, code, reasoning, tool-calling) that improves downstream accuracy and lowers KL divergence.

Quality and Mini use a 3-tier gradient. Balanced and Compact use a simpler 2-tier gradient (edge vs. middle) — their "Near-Edge" and "Middle" columns are the same precision.

MTP Head

The bundled MTP head (blk.40.* including the nextn.* projection + norms) is quantized to Q8_0 (near-lossless) for high draft accuracy. Enable with:

llama-server -m ornith-1.0-35b-APEX-...-MTP.gguf --spec-type draft-mtp

Usage Examples

llama.cpp server (basic)

llama-server \
  -m ornith-1.0-35b-APEX-I-Compact.gguf \
  -ngl 99 \
  -c 32768 \
  --flash-attn on \
  --temp 0.6 \
  --top-p 0.95

With self-speculative decoding (MTP variants)

llama-server \
  -m ornith-1.0-35b-APEX-I-Compact-MTP.gguf \
  --spec-type draft-mtp \
  -ngl 99 \
  -c 32768 \
  --flash-attn on

llama.cpp server with vision

Ornith has a built-in vision encoder. Vision support in GGUF format is experimental — if a compatible mmproj becomes available, pass it with --mmproj.

Hardware Notes

Acknowledgements

• —Base model: DeepReinforce — Ornith-1.0-35B (MIT)
• —APEX quantization: LocalAI team
• —MTP donor tensors: IHaveNoClueAndIMustPost
• —Calibration dataset: v3+ultrachat (pile-10k, GSM8K, eaddario imatrix-calibration, HuggingFaceH4/ultrachat_200k)
• —Built on llama.cpp