julianmb/Ornith-1.5-35B-A3B-ROCmFP4-GGUF

Ornith-1.5-35B-A3B ROCmFP4

ROCmFP4 quantization of Ornith 1.5 35B-A3B for AMD Strix Halo (gfx1151) and RDNA 3.5 GPUs, produced with the Q4_0_ROCMFP4_STRIX_LEAN preset (FP16 embedding/norm preservation, nextn MTP head kept at q8_0).

Revision 2026-08-28: rebuilt from the ornith-ai aug-24 MTP refresh. the retrained MTP head now verifies at 87.98% draft acceptance (mean 3.70/4 positions) for 105.6 tok/s effective decode on strix halo — speculative decoding is now recommended. note: the new MTP head requires this base revision; do not graft it onto older quants.

Quantization Details

Measured Performance (AMD Ryzen AI Max+ 395, Radeon 8060S, Mesa RADV Wave64)

ROCmFP4 is ~7.5% faster and 16.7% smaller than stock Q4_K_M; with MTP speculative decoding it reaches ~137% of bare decode.

Serving

HaloFPX (Text + Vision)

halofpx pull downloads and verifies both the ROCmFP4 weights and BF16 vision projector:

halofpx pull ornith-1.5-35b
halofpx serve
halofpx load ornith-1.5-35b

halofpx list reports model-weight and vision-projector readiness separately.

Direct llama-server Vision

llama-server \
  -m Ornith-1.5-35B-A3B-ROCmFP4.gguf \
  -mm mmproj-Ornith-1.5-35B-BF16.gguf \
  -ngl 99 -c 131072 -fa on --no-mmap

Full 256K Max Context (Strix Halo 128GB UMA):

llama-server \
  -m Ornith-1.5-35B-A3B-ROCmFP4.gguf \
  -ngl 99 -c 262144 -fa on --no-mmap

Multi-Slot / Conservative Memory (131K Context):

llama-server \
  -m Ornith-1.5-35B-A3B-ROCmFP4.gguf \
  -ngl 99 -c 131072 -fa on --no-mmap

Note: speculative decoding (MTP, --spec-type draft-mtp) is recommended on this model with the aug-24 refresh: measured 87.98% draft acceptance (mean 3.70/4 positions) for 105.6 tok/s effective decode vs 76.9 bare. suggested profile: --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.6 -fa on.