zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated-MTP-NVFP4

Qwen3.8-27B-Dominatrix-abliterated — MTP-NVFP4 (DFlash 2 compatible)

Mixed-precision NVFP4/FP8 PTQ of [zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated](https://huggingface.co/zebulon-prime/Qwen3.8-27B-Dominatrix-abliterated) — allura-org's Dominatrix roleplay finetune with huihui-ai's refusal direction projected out.

23 GB, built with NVIDIA TensorRT Model Optimizer for Blackwell (SM120) inference in SGLang or vLLM.

The distinguishing feature: `lm_head` is left dense BF16, which is a hard requirement for DFlash 2 speculative decoding. Most ModelOpt NVFP4 exports of this architecture quantize lm_head and therefore cannot run DFlash 2 at all. Cost of the dense head is ~1.9 GB of VRAM over a packed one.

Quantization layout

Export format is ModelOpt MIXED_PRECISION with a per-layer map in hf_quant_config.json. Calibrated on in-domain ChatML roleplay text rather than a generic news corpus.

hf_quant_config.json records producer.version: 0.0.0 because it was built from an editable install. That field is not meaningful provenance.

Serving

SGLang with DFlash 2

Requires the z-lab/Qwen3.8-27B-DFlash2 drafter and an SGLang build including PR #35371.

sglang serve \
  --trust-remote-code \
  --model-path /models/Qwen3.8-27B-Dominatrix-abliterated-MTP-NVFP4 \
  --mem-fraction-static 0.70 \
  --attention-backend flashinfer \
  --chunked-prefill-size 2048 \
  --reasoning-parser qwen3 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path /models/Qwen3.8-27B-DFlash2-zlab \
  --speculative-dflash-block-size 8 \
  --speculative-draft-model-quantization unquant

--kv-cache-dtype can be omitted: this checkpoint declares kv_cache_quant_algo: FP8, so SGLang's auto resolves it.

Without speculative decoding

Drop the four --speculative-* flags. The dense lm_head buys nothing in that configuration but is otherwise harmless.

MTP

The mtp.* tensors survive the quant in BF16, so MTP speculation remains available as an alternative drafter. Pick one — MTP or DFlash 2, not both.

Hardware

NVFP4 requires Blackwell (SM120+) for native FP4 tensor-core execution. Built and tested on an RTX PRO 6000 Blackwell. Weights are ~23 GB, leaving room for a long-context KV cache and the 2B DFlash 2 drafter.

Quality

Fidelity of the underlying BF16 abliteration versus stock Dominatrix is summarised on the BF16 card.

*The quantization error of this build has not been measured.* Treat it as unquantified. Leaving lm_head dense should help, since the output projection is among the most quantization-sensitive layers, but that is reasoning, not a measurement.

Sampler guidance from upstream Dominatrix carries over: temperature 1.0–1.25 with min_p 0.1 or top_p 0.95; some prefer 0.7 and nothing else.