Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF

Qwen3.8-27B-SSMFIX-UD-Q3KXL-GGUF

中文说明

Experimental community conversion. This is not an official Qwen release.

This repository contains a single GGUF text-inference model:

Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf

It is derived from the repaired BF16 model redashes/Qwen3.8-27B-BF16-SSMFIX, which in turn is derived from the official Qwen/Qwen3.8-27B.

At a glance

UD-Q3_K_XL is a quantization-tier name, not a claim that every tensor is stored as pure Q3_K. The exact mixed recipe is documented below and in quantization_recipe.txt.

Model lineage and attribution

Qwen/Qwen3.8-27B
        |
        v
redashes/Qwen3.8-27B-BF16-SSMFIX
        |
        v
Luis23333/Qwen3.8-27B-SSMFIX-UD-Q3_K_XL-GGUF

• —Upstream model: Qwen/Qwen3.8-27B
• —Direct BF16 source: redashes/Qwen3.8-27B-BF16-SSMFIX
• —Background discussion: Qwen3.8-27B discussion #76
• —GGUF publisher: Luis23333

The direct source model describes itself as an independent verification of a community investigation into abnormal SSM/conv1d scale behavior. This GGUF preserves that repaired source lineage and then applies the quantization recipe described below. It should be treated as an experimental community artifact, not as an official correction to Qwen3.8.

What SSMFIX changes

The BF16 source model applies per-layer alpha rescaling to eight ssm_conv1d.weight tensors. The documented correction factors are:

In the local quantization log, ssm_conv1d.weight tensors remain F32 in the GGUF conversion. The repair is therefore not deliberately quantized away by the recipe.

Quantization recipe

The model was quantized from a BF16 GGUF generated from the repaired source weights using llama.cpp build 9222 (9a532ae4b) and an importance matrix named imatrix_unsloth.gguf_file.

The exact tensor overrides are recorded in quantization_recipe.txt. The resulting file is a custom mixed quantization, not a pure single-type Q3_K file.

Important modality limitation

The upstream Qwen3.8 model is a vision-language model. This repository contains only the language-model GGUF produced by the conversion above. It does not include an mmproj/vision projector file, so this single artifact is documented and tested for text inference only. Use the upstream Transformers repository or a compatible model-plus-projector package when image or video input is required.

Usage with llama.cpp

Use a recent llama.cpp build with Qwen3.8/Qwen3.5 architecture support.

Interactive text chat

llama-cli \
  -m Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf \
  --jinja \
  --reasoning on \
  -cnv

Disable thinking with --reasoning off when supported by your llama.cpp build.

HTTP server

llama-server \
  -m Qwen3.8-27B-BF16-SSMFIX-UD-Q3_K_XL.gguf \
  --jinja \
  --reasoning auto \
  --ctx-size 16384 \
  --n-gpu-layers auto \
  --fit on \
  --flash-attn auto

The 262,144-token context length is inherited metadata, not a guarantee that a particular GPU can allocate that context. A 16 GiB GPU should start with a smaller context such as 16,384 and increase it only after measuring memory use.

The upstream Qwen recommendations are approximately:

• —Thinking: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0.
• —Non-thinking: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5.

Evaluation and local smoke tests

Reference results from the BF16 source

The following results belong to the direct BF16 source model, not to this GGUF. They are included only as provenance context and should not be interpreted as a GGUF quality benchmark.

GGUF runtime smoke test

Local text-only smoke tests were run with llama.cpp build 9222 (9a532ae4b) on an NVIDIA RTX 5070 Ti, with context size 16,384, six prompts, three runs per prompt, and a maximum of 256 generated tokens.

These are runtime smoke-test measurements, not broad capability scores. In this local setup MTP2 was slower than baseline, so this repository does not claim that speculative decoding is faster for this artifact.

Limitations and disclaimer

• —This is an experimental community conversion and is not an official Qwen release.
• —The SSMFIX hypothesis and its claimed benefits are not guaranteed for every workload.
• —The GGUF has not been assigned the BF16 source's quality scores.
• —This upload contains no vision projector and is not presented as a single-file multimodal package.
• —Long-context behavior depends on the runtime, KV-cache settings, available VRAM/RAM and prompt shape.
• —Validate the model on your own workload before relying on it in production.

License

The model follows the Apache-2.0 license of the upstream Qwen release. See LICENSE. Please preserve the upstream attribution and the direct-source attribution when redistributing this derived artifact.