ninfer-5080/Qwen3.8-27B-RTX5080

hf download ninfer-5080/Qwen3.8-27B-RTX5080 \
  qwen3_8_27b.ninfer \
  --local-dir .

Verify the artifact:

sha256sum qwen3_8_27b.ninfer

Expected:

c4a7e9ab593a7f42d58208fa0065d67a82d61921107686cc9f6ed1ec6b050e21  qwen3_8_27b.ninfer

A matching SHA-256 identifies the exact validated project artifact regardless
of the filename or the machine from which it was downloaded.

Model artifact vs runtime version

The model artifact and the NInfer runtime are versioned independently.

The artifact currently published here has remained byte-identical across
multiple later runtime optimizations. A newer NInfer runtime therefore does
not imply that a new .ninfer model file is required.

The canonical artifact identity is:

bytes:
16461267456

SHA256:
c4a7e9ab593a7f42d58208fa0065d67a82d61921107686cc9f6ed1ec6b050e21
Validated v1.3 production runtime

Validated source commit:

ceb32f7d002edab224a83a2e2609f45fca4f8919

Validated ninfer-serve SHA-256:

3179bfbcb88a72c04b983f28c25c62db468fbc8ef267fe043899de30a4281c56

v1.3 adds:

corrected Q4/Q4 strided attention output handling
server-wide default thinking-budget support
rolling tool checkpoints for agent/tool-loop workloads
preservation of the validated 131K/Q4-KV/MTP-3/Vision profile

Full release record:

https://github.com/toddballinger/ninfer-5080/blob/main/docs/RELEASE_QWEN3.8_27B_RTX5080_V1.3.md

Recommended serving profile

For more GPU-memory headroom, Vision 1792 is the recommended general profile:

./ninfer-serve qwen3_8_27b.ninfer \
  --host 0.0.0.0 \
  --port 8080 \
  --model-id qwen3.8-27b \
  --max-context 131072 \
  --kv-capacity 131072 \
  --prefill-chunk 896 \
  --kv-dtype q4 \
  --spec mtp \
  --draft-tokens 3 \
  --no-cuda-graph \
  --max-concurrency 1 \
  --default-thinking-budget 2048 \
  --prefix-checkpoint-policy rolling-tool \
  --vision \
  --vision-max-tokens 1792

Measured startup margin:

Vision workspace       115.7751 MiB
Free after startup      26.56 MiB
Planned slack           28.88 MiB
Maximum validated Vision profile

Vision 2048 is also validated and is the profile used by the v1.3 production
OpenClaw deployment:

--vision-max-tokens 2048

Measured startup margin:

Vision workspace       132.3142 MiB
Free after startup       8.56 MiB
Planned slack           10.08 MiB

This profile is intentionally tight. Use a clean GPU.

True-128K validation

The project does not describe a configuration as "true 128K" merely because
the configured maximum is 131,072.

The qualification workload contains an actual 118,001-token prompt while
retaining a full 131,072-token KV allocation.

A qualified feature-complete runtime produced:

Metric	Result
Prompt tokens	118,001
Max context	131,072
KV capacity	131,072
Prefill	1378.85 tok/s
Decode	71.44 tok/s
MTP acceptance	44.74%
MTP acceptance length	2.31 tok/round

A later Q5 A16 LinearAdd semantic-port qualification produced 1376.30 tok/s
prefill and 71.53 tok/s decode on the same workload while retaining the same
model SHA.

Benchmark results are commit-scoped; see the GitHub validation ledger rather
than treating any one result as a floating "current" benchmark.

Multimodal validation

The final HostMapped Vision path has been validated for:

deterministic image understanding
deterministic video understanding
multi-image conversation history
cached historical-media accounting
coexistence with the full 131,072 text context/KV allocation

A synthetic red/blue image was correctly identified by side, and a deterministic
red → green → blue video was returned in the correct chronological order.

Details:

https://github.com/toddballinger/ninfer-5080/blob/main/docs/VISION_128K.md

Model sources

Target model:

Qwen/Qwen3.8-27B
revision:
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0

DFlash2 source:

z-lab/Qwen3.8-27B-DFlash2
revision:
50307d4c4cde6860d4eee73e2547cd786fe8e8a4

Both upstream Hugging Face repositories currently declare Apache-2.0 licensing.

Quantization profile

Main text-core distribution:

Format	Share
Q3G64_F16S	42.42%
Q4G64_F16S	45.92%
Q5G64_F16S	11.57%
BF16 / FP32	~0.10%

Effective main-model quantization:

~3.953 BPW
Reproducibility

This Hugging Face repository currently contains the validated reference
artifact.

A CPU-only GitHub Actions conversion/publishing workflow is being integrated
separately. Before any automated build is allowed to replace this artifact while
claiming byte-identical reproduction, it should reproduce both:

SIZE:
16461267456

SHA256:
c4a7e9ab593a7f42d58208fa0065d67a82d61921107686cc9f6ed1ec6b050e21

This prevents build automation from silently replacing a known-good model with
a different artifact.

Documentation
Project overview:
https://github.com/toddballinger/ninfer-5080
Validated manifest:
https://github.com/toddballinger/ninfer-5080/blob/main/docs/VALIDATED_MANIFEST.md
v1.3 release:
https://github.com/toddballinger/ninfer-5080/blob/main/docs/RELEASE_QWEN3.8_27B_RTX5080_V1.3.md
Vision:
https://github.com/toddballinger/ninfer-5080/blob/main/docs/VISION_128K.md
Reproducibility:
https://github.com/toddballinger/ninfer-5080/blob/main/docs/REPRODUCIBILITY.md
Benchmarks:
https://github.com/toddballinger/ninfer-5080/blob/main/docs/BENCHMARKS.md
Memory profile:
https://github.com/toddballinger/ninfer-5080/blob/main/docs/MEMORY_PROFILE.md
Credits

This project builds on the work of the NInfer project and the Qwen/DFlash2
ecosystem.

NInfer upstream: Neroued and contributors
Qwen3.8-27B: Qwen team
DFlash2: z-lab / project contributors
RTX 5080 optimization, validation and release profile: Todd Ballinger
Low-memory automated conversion / CI work in progress: starskyzheng

Please preserve applicable upstream copyright, attribution and license notices.