Modelpublic

Frosty40/Ornith-1.0-35B-GGUF-NVFP4

sourceHugging Facemitupdated 3mo agoView on Hugging Face

1likes91downloads

Model Card

Ornith-1.0-35B — GGUF / NVFP4

Status: R&D. data release — no claims.

GGUF NVFP4 (W4A4) build of deepreinforce-ai/Ornith-1.0-35B (qwen3_5_moe, 35B MoE), converted from the sakamakismile/Ornith-1.0-35B-NVFP4 compressed-tensors checkpoint for llama.cpp. Text-only (the source vision tower is not included).

—File: ornith-1.0-35b-GGUF_NVFP4.gguf — 21.0 GB (21,046,365,120 bytes)
—SHA256: d9ab37f61fedc3d75bac7516baf45da383fbd9bc4ff1cf240962ca1bc31b3177
—Tensors: 1593 (NVFP4 390, F32 1161, F16 2) · 40 layers · 256 experts (8 active) · hybrid linear/full attention

Runtime

Requires a llama.cpp build with `qwen35moe` architecture + NVFP4 support (not in mainline as of this release). Tested binary: b1-1ec44d1, Linux aarch64, CUDA sm_121, NVIDIA GB10.

llama-cli -m ornith-1.0-35b-GGUF_NVFP4.gguf --jinja --reasoning-budget 0 -ngl 999 -p "..."

Benchmarks (this GGUF, GB10)

Accuracy

Benchmark	Score	n	Method
GSM8K	97.0%	200	generative, chat, thinking-off, exact-match
HellaSwag	82.1%	1000	loglikelihood
Winogrande	71.6%	1000	loglikelihood
ARC-Challenge	49.2%	299	loglikelihood, multiple-choice
MMLU	41.1%	2000	0-shot loglikelihood, multiple-choice
TruthfulQA-MC1	35.7%	817	loglikelihood, multiple-choice
Wikitext-2	PPL 6.36	100 chunks	`-c 2048`

MC/loglikelihood via llama-perplexity; GSM8K via llama-server (--reasoning-budget 0). 0-shot, no few-shot priming.

[image]

Throughput / concurrency (llama-batched-bench, 512-tok prompt + 256-tok gen)

Agents	Prompt t/s	Agg gen t/s	Per-agent gen t/s	Total t/s	256-tok reply (s)
1	2333	69.0	69.0	195	3.7
8	2372	206.5	25.8	528	9.9
16	2356	225.7	14.1	568	18.1
32	2353	277.1	8.7	673	29.6
64	2351	326.3	5.1	766	50.2
96	2356	342.6	3.6	796	71.7
128	2353	350.6	2.7	810	93.5
192	2358	361.5	1.9	830	136.0
256	2356	372.3	1.5	849	176.0

Generation saturates ~370 t/s; 256 agents ran without OOM (~40 GB used). Compute-bound, not memory-bound.

[image] [image]

Raw logs + CSVs: benchmarks/.

Conversion

convert_hf_to_gguf.py --outtype f16 --fuse-gate-up-exps --no-mtp (llama.cpp qwen35moe converter). Source safetensors SHA256 ec25ada854768610c7b841889b2a1b9f06163eda798a296401be87d0a2613cdd.

Credits

—Base model: DeepReinforce — deepreinforce-ai/Ornith-1.0-35B (MIT)
—NVFP4 quantization: sakamakismile/Ornith-1.0-35B-NVFP4 (llm-compressor, compressed-tensors)
—Architecture: Qwen3.5-MoE (Qwen / Alibaba)
—GGUF format + runtime: llama.cpp / ggml

License

MIT, inherited from the base model. This repository adds only the GGUF/NVFP4 weights, conversion metadata, and benchmark data.