Modelpublic

Hal0ai/Qwen-AgentWorld-Hal0-35B-A3B-ROCmFP4

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes38downloads

Model Card

Qwen-AgentWorld-Hal0-35B-A3B-ROCmFP4

A ROCmFP4 (Q4_0_ROCMFP4_STRIX_LEAN, ~4.29 bpw) GGUF quant of Qwen/Qwen-AgentWorld-35B-A3B — a 35B-A3B Mixture-of-Experts (qwen35moe) world-model, 262K context.

Built for AMD Strix Halo (gfx1151) unified-memory inference on the hal0 agent platform.

⚠️ Requires the ROCmFP4 llama.cpp fork — not upstream

ROCmFP4 is an experimental, fork-specific quantization format (UE4M3-scale FP4). It is not loadable by stock llama.cpp or standard GGUF tooling — those will fail with 101 is not a valid GGMLQuantizationType.

Run it with either:

—the hal0 Strix Halo toolbox image ghcr.io/hal0ai/amd-strix-halo-toolboxes:rocm-7.2.4-rocmfp4-server, or
—a build of the `rocmfp4-llama` fork (branch mtp-rocmfp4-strix), targeting gfx1151.

Details


Architecture	`qwen35moe` (35B total / ~3B active, MoE)
Quant	`Q4_0_ROCMFP4_STRIX_LEAN` (~4.38 bpw target; 4.29 bpw measured)
Recipe	ROCmFP4 experts/FFN + Strix K/V + Q5_K token embeddings
File size	~17.3 GiB (from 66 GiB BF16)
MTP	No (plain quant, no speculative-decode draft head)
Context	262144
Base	Qwen/Qwen-AgentWorld-35B-A3B (Apache-2.0)

Provenance

Quantized from the BF16 GGUF in unsloth/Qwen-AgentWorld-35B-A3B-GGUF, using that repo's importance matrix (imatrix_unsloth.gguf), via the hal0 ROCmFP4 quantize pipeline (llama-quantize --imatrix ... Q4_0_ROCMFP4_STRIX_LEAN).