kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16

Fable-Fusion 711 for vLLM (NVFP4A16)

This is the vLLM version of DavidAU's Qwen3.6-27B Fable-Fusion 711. The weights are FP4. Everything sensitive stays BF16. The repo is 28 GB instead of 56. It serves as an OpenAI-compatible endpoint on any NVIDIA GPU with about 32 GB of VRAM, Ampere or newer. Blackwell is not required.

Quickstart

vllm serve kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16 \
  --max-model-len 32768 --gpu-memory-utilization 0.85

Tested on vLLM 0.26.0. The server selects the Marlin NVFP4 kernel and warns that the GPU lacks native FP4 compute. The warning is expected: activations are BF16 by design, so the FP4 tensor-core path does not apply.

For faster decoding, enable the built-in MTP drafter:

vllm serve kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16   --max-model-len 32768 --gpu-memory-utilization 0.85   --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":5}'

DavidAU tuned this model for specific sampler settings. They translate to the OpenAI API as follows:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

# Thinking mode (default template): creative work, reasoning
r = client.chat.completions.create(
    model="kkuspa/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4A16",
    messages=[{"role": "user", "content": "Open a noir story set in a lighthouse."}],
    temperature=1.0, top_p=0.95, max_tokens=2048,
    extra_body={"top_k": 20, "min_p": 0.0, "repetition_penalty": 1.0},
)

# Coding: temperature=0.6, same top_p and top_k
# Instruct mode: temperature=0.7, top_p=0.80, presence_penalty=1.5

The repo ships two chat templates. chat_template.jinja (thinking) is the default. For instruct-style responses without a thinking block, pass --chat-template pointing at chat_template-instruct.jinja at serve time.

This repo or the GGUF?

• —llama.cpp, LM Studio, Ollama, one user on a consumer card: use DavidAU's GGUF repo.
• —vLLM, concurrent requests, an OpenAI-compatible server, or 100k+ context: use this repo.

At the time of publication this is the only quant of Fable-Fusion 711 that stock vLLM can serve. The other safetensors quants of this model use a custom runtime format.

Fidelity

Measured with lm-evaluation-harness 0.4.12 (vLLM backend, 0-shot, batch auto) against the BF16 parent on the same hardware:

Deltas sit inside the standard error on every task (ARC-Challenge stderr is 0.014). The quantization is a data-free RTN pass. No calibration dataset was used, so nothing was tuned toward any benchmark. Absolute scores depend on harness settings; the point of this table is the like-for-like comparison, both configs measured 0-shot on the same harness and hardware.

Speed

Single request, 512-token greedy generations, 1x RTX PRO 6000 Blackwell (96 GB), vLLM 0.26.0:

Acceptance across the test: 996 of 3000 drafted tokens (about 1.7 extra tokens per draft window). Throughput under concurrent load will differ.

What was quantized

llm-compressor 0.12.0, one-shot:

QuantizationModifier(
    targets="Linear",
    scheme="NVFP4A16",
    ignore=["lm_head", "re:.*visual.*", "re:.*linear_attn.*", "re:.*mtp.*"],
)

Only the Linear weights in the full-attention and MLP blocks carry FP4. The DeltaNet linear-attention layers, the vision tower, the MTP drafter, and lm_head keep BF16, because hybrid-attention layers and drafters degrade badly under 4-bit quantization. The exact recipe ships in recipe.yaml.

Packaging notes

• —The parent repo ships no image-processor configs. preprocessor_config.json and video_preprocessor_config.json here come from Qwen/Qwen3.6-27B.
• —The 15 MTP drafter tensors live in model-mtp-bf16.safetensors and are wired into model.safetensors.index.json. Transformers drops them on load; they were re-extracted from the parent checkpoint so speculative decoding remains possible.

Content notice

The parent model is refusal-ablated ("Uncensored", "Heretic") and tuned for unconstrained creative writing. This quant changes none of that. It will produce content that aligned models refuse. You are responsible for how you deploy it.

Caveats

• —The vision path loads but has not been benchmarked.

Lineage and attribution

Qwen/Qwen3.6-27B by the Qwen team, then the Fable-Fusion 711 multi-stage tune and merge by DavidAU with collaborators credited on the parent card, then this quantization. Thanks to all of them. Apache-2.0 throughout.