Modelpublic

Kymidnight/Qwen3.5-122B-A10B-ROCMFP4-MTP-GGUF

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

1likes263downloads

Model Card

Qwen3.5-122B-A10B-ROCMFP4-MTP-GGUF

The first ROCmFP4 MTP quantization of Qwen3.5-122B optimized for AMD Strix Halo (Ryzen AI 300 series) APUs.

Key Features

—Format: GGUF with integrated MTP (Multi-Token Prediction) head
—Quantization: ROCmFP4 (custom 4-bit quantization for AMD GPUs)
—Architecture: A10B (Adaptive 10/40 layer hybrid attention for 128K context)
—Context: 256K tokens
—MTP Acceptance: 92-95%

Benchmarks (Strix Halo 128GB, Aug 25 2026)

Tested with llama.cpp ROCmFPX b257, Vulkan backend, MTP enabled.

Prompt Size	Prompt Speed	Gen Speed
36 tokens	91.5 t/s	43.8 t/s
71 tokens	99.2 t/s	39.3 t/s
932 tokens	379.6 t/s	41.0 t/s

—Generation: ~41-44 t/s (consistent across prompt sizes)
—Prompt processing: up to 380 t/s (large prompts saturate GPU bandwidth)
—MTP accept rate: 94%
—Model size: ~65 GB (5 shards)

Usage

bash

llama-server \  --model Qwen3.5-122B-A10B-ROCMFP4-STRIX-00001-of-00002-00001-of-00005.gguf \  --port 8080 \  -c 262144 \  --n-gpu-layers 99 \  --spec-type draft-mtp \  --spec-draft-n-max 2 \  --spec-draft-n-min 1 \  --spec-draft-p-min 0.75 \  --spec-draft-type-k q8_0 \  --spec-draft-type-v q8_0

Notes

—Built with llama.cpp b10615+ (ROCmFPX b257)
—Requires AMD ROCm 6.2+ or Vulkan backend
—MTP head is bundled in the GGUF (no separate draft model needed)
—Optimized for 128GB+ VRAM systems
—Hybrid attention (10/40 full layers) enables 256K context in 128GB VRAM

License

Apache 2.0

Generated by Tom Yoder | August 2026