Modelpublic

qualcomm-ai-hub-community/Qwen3-4B-Instruct-2507-Hi-Fi-fraQtl

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

1likes830downloads

Model Card

Qwen3-4B-Instruct-2507 Hi-Fi GGUF — fraQtl

Calibration-aware GGUF builds of Qwen/Qwen3-4B-Instruct-2507: byte-matched to the leading community Q4KM with measurably lower divergence from the original weights, retrieval-verified through the full 262,144-token native context — plus a Q4_0 edition in the NPU-native fixed format.

Model Conversion Contributor: fraQtl — built by fraQtl

Model Stats:

—Quantization Type: calibration-aware per-tensor Q4KM-class (Hi-Fi) and Q4_0 (fixed format) — weights only, GGUF
—Calibration: fraQtl code+math imatrix (included in this repo)
—Maximum context length: 262,144 (native), retrieval-verified below
—Runtime: llama.cpp (CPU/GPU). GGUF works with GenieX unmodified — per the AI Hub team. (NPU/Hexagon execution not claimed — not yet verified.)

Model Download

Artifact	Size	Class	Target Runtime	File
Hi-Fi (max fidelity)	2.50 GB	Q4KM-class	llama.cpp CPU/GPU · GenieX	`Qwen3-4B-Instruct-2507-fraQtl-HiFi-Q4_K_M.gguf`
Hi-Fi Q4_0	2.38 GB	Q4_0 (NPU-native format)	llama.cpp CPU/GPU · GenieX	`Qwen3-4B-Instruct-2507-fraQtl-HiFi-Q4_0.gguf`

Fidelity receipts (KLD vs the ORIGINAL model, lower is better)

Symmetric top-20 KLD vs a Q8 teacher quantized from the pinned original bf16; identical binary, template, and slices for every arm; 3 runs, drift 0:

Arm	Bytes	KLD code/math ↓	KLD general ↓
fraQtl Hi-Fi Q4_K_M (this repo)	2,496,879,712 (−0.016%)	0.025317 (−55.9%)	0.033753 (−46.4%)
MaziyarPanahi Q4KM (canonical, credited)	2,497,280,448	0.057372	0.063008
unsloth Q4KM (credited)	2,497,281,120	0.030018	0.042626

Long-context retrieval: needle exact-match, 3 depths × 3 keys per context — 18/18 at 8K+32K (×3 runs) and 9/9 at the full 262,144-token native context (×2 runs); per-cell receipts in the canonical repo.

Q4_0 — calibration lever isolated (fixed format)

Arm	KLD code/math ↓	KLD general ↓
fraQtl Q4_0 (our imatrix)	0.059665	0.080458
naive Q4_0 (no imatrix, same source)	0.070135	0.088598
GenieX-designated Q4_0 (unsloth's build, credited)	0.060996	0.077572

Calibration alone: −14.9% code/math, −9.2% general at identical format and source. Vs the GenieX-designated universal asset (a well-made build): we are −2.2% on code/math and +3.7% behind on general — both directions stated. K-quant-on-Hexagon NPU acceleration is unverified by us; the Q4_0 artifact is the NPU-native-format option.

Canonical repo + full receipts

Complete receipts (7 JSONs) and a runnable receipts pack (single-file runner on stock llama.cpp + teacher logit dumps — re-run every number above against any GGUF of this model): fraQtl/Qwen3-4B-Instruct-2507-Hi-Fi-GGUF

License

Apache-2.0 (inherited from the base model).