pyros-vault/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NInfer

Qwen3.8-27B Cold Fusion GAIN V1.1 for NInfer-4090

This repository contains a single-file, mixed-precision .ninfer conversion of DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1, built for UDPSendToFailed/ninfer-4090 on a 24 GB RTX 4090 (sm_89).

This is a deployment-format conversion, not additional training. The model weights come from the direct upstream snapshot; three frontend files were replaced with NInfer's registered Qwen3.8 versions as documented below. This card does not independently validate every quality and reduced-thinking claim made upstream. The artifact is not a Safetensors or GGUF checkpoint and is not intended for Transformers, llama.cpp, or unrelated NInfer forks.

Quick facts

The conversion report records NInfer revision `6d3fd16`. The verification below used the later source revision `e4233d6`, whose cached-token reporting change is merged upstream in `UDPSendToFailed/ninfer-4090#5`.

What is inside the .ninfer bundle

The file is a self-contained runtime bundle: mixed-precision model weights plus six embedded frontend resources (tokenizer.json, tokenizer and generation configs, chat_template.jinja, and image/video processor configs). Q4G64_F16S, for example, means 4-bit weights quantized in groups of 64 with FP16 scales; the Q5/Q6 variants use the same group size, while W8G32_F16S uses 8-bit weights in groups of 32.

NInfer materializes only the components selected at startup. In the verified runs, resident weights were 15.92 GiB for baseline text, 16.19 GiB with Vision, and 16.67 GiB with MTP. The 16.96 GiB artifact size therefore is not the same thing as per-mode VRAM usage; KV state, workspaces, CUDA Graphs, and concurrency add further runtime memory.

Frontend compatibility note

NInfer strictly validates embedded frontend resources against its registered Qwen3.8 contract. The direct snapshot's tokenizer.json, tokenizer_config.json, and generation_config.json differed, so the converter embedded the official registered Qwen3.8 versions instead. The direct snapshot's chat template and image/video processor configs already matched. This substitution affects tokenization and generation defaults, not the converted Cold Fusion weight tensors, and is why this deployment artifact should be used through the linked NInfer runtime.

Upstream model

DavidAU describes the direct source as a light Qwen3.8-27B Cold Fusion tune combining GAIN and Unsloth training, aimed at maintaining detail while reducing thinking-token use. Those are upstream claims; consult the source model card for its training narrative, evaluations, reasoning examples, and GGUF links. The local checks below validate the converted artifact's loading and inference paths, not the full upstream benchmark suite.

Download

hf download pyros-vault/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NInfer `
  --local-dir .\Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NInfer

Verify the artifact:

Get-FileHash `
  '.\Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NInfer\Qwen3.8-27B-Cold-Fusion-GAIN-V1.1.ninfer' `
  -Algorithm SHA256

Serving with the intended NInfer runtime

Build a current native sm_89 version of `UDPSendToFailed/ninfer-4090`. .ninfer is a registered, model-bound format; release binaries or other forks without the matching Qwen3.8 target may not load this file.

The following serving examples use the same conservative memory profile as the CLI verification below: a 4,096-token KV capacity with CUDA Graphs disabled. Sampling and thinking behavior remain request-configurable.

Text with MTP4

.\ninfer-serve.exe '.\Qwen3.8-27B-Cold-Fusion-GAIN-V1.1.ninfer' `
  --kv-dtype rk4v4-e8 `
  --spec mtp --draft-tokens 4 --lm-head-draft `
  --max-context 4096 --kv-capacity 4096 --prefill-chunk 512 `
  --preserve-thinking --no-cuda-graph

For baseline decoding, omit --spec mtp --draft-tokens 4 --lm-head-draft. With the interactive ninfer.exe CLI, use --no-thinking for short non-reasoning output or --reasoning-effort low|medium|xhigh when thinking is enabled. With ninfer-serve, select the effort in request JSON as reasoning_effort for Chat Completions or reasoning.effort for Responses.

Vision with MTP4

.\ninfer-serve.exe '.\Qwen3.8-27B-Cold-Fusion-GAIN-V1.1.ninfer' `
  --vision --vision-max-tokens 1024 `
  --kv-dtype rk4v4-e8 `
  --spec mtp --draft-tokens 4 --lm-head-draft `
  --max-context 4096 --kv-capacity 4096 --prefill-chunk 512 `
  --preserve-thinking --no-cuda-graph

The server exposes OpenAI-compatible /v1/chat/completions and /v1/responses endpoints plus Anthropic-compatible /v1/messages at http://127.0.0.1:8080. Prefix reuse is enabled by default; current upstream builds expose it as usage.prompt_tokens_details.cached_tokens.

This Qwen3.8 artifact supports MTP with one to five draft tokens. It does not contain a DFlash model; NInfer's DFlash path is specific to the 35B-A3B target.

Local verification results

These are bounded single-run integration measurements, not a benchmark suite. Tests used greedy decoding, thinking disabled, rk4v4-e8 KV, --max-context 4096, --kv-capacity 4096, --prefill-chunk 512, and --no-cuda-graph on one RTX 4090.

The baseline and MTP rows used the same prompt and produced the same 99-token answer, including the same Fibonacci implementation and assertions. This makes their decode-rate ratio meaningful for this one workload, but not a general performance guarantee.

The pinned NInfer fixture (image, message JSON) contains the title NIFER VISION 731, three red circles, and a blue square to the left of a green triangle. Its short decode rate is included for completeness and should not be compared directly with the coding run.

Conversion details

The dedicated converter preflighted 1,199 BF16 source tensors across 18 shards and emitted 1,118 runtime tensors plus six embedded tokenizer/template/processor resources. The 1,124 stored objects use a mixture of BF16, FP32, groupwise Q4/Q5/Q6, and W8 formats. This is a quantized deployment artifact, not a lossless copy of the BF16 checkpoint.

Conversion completed in 69.15 seconds on the RTX 4090. The complete manifest and environment are retained in Qwen3.8-27B-Cold-Fusion-GAIN-V1.1.ninfer.conversion.json.

The converter report records the local source directory but not its Hugging Face revision. The adjacent Hugging Face download metadata pins the converted source files to commit `b81b04dfaa66db4e60d73eb6f144c8e99a6cce23`. The frontend substitution above is also reflected by the embedded resource hashes in the conversion report.

Important limitations

• —Only the documented 4k profile is locally verified here. The upstream config declares 262,144 positions, but this card does not claim a long-context validation.
• —Vision is opt-in. NInfer defaults to an 8,192-token Vision scratch capacity; start with --vision-max-tokens 1024 on a 24 GB card and increase only when needed.
• —Quantization can change outputs. Quality and performance depend on the prompt, sampler, KV dtype, context length, and runtime revision.
• —Upstream claims remain upstream claims. The local checks validate deployment functionality rather than the complete Cold Fusion training methodology or benchmark suite.
• —No hosted inference. The Hugging Face Inference API cannot execute .ninfer files.

License and credits

The direct source and Qwen base declare Apache-2.0. A copy of the license is included in `LICENSE`. Users remain responsible for reviewing upstream terms and complying with applicable laws.

NInfer runtime code is not redistributed here; obtain it from its separately licensed upstream repository.

Converted, tested, and packaged by pyros-vault.