AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Text-NVFP4-MTP

[!IMPORTANT] Superseded by Qwen3.8 — use this instead: `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4-MIXED` Operator recipes (Spark Dynamic DFlash lattice, TP=2, RTX): `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED` This Qwen3.6-27B card stays online for history and existing downloads. For new work, move to the Qwen3.8 Ultimate Uncensored NVFP4-MIXED release. That line is far superior in capability and reaches substantially more of what the architecture can do, with a more refined mixed quantization methodology (NVFP4 + FP8 lattice) than these older Qwen3.6 exports. Serve with the unified container: ghcr.io/aeon-7/aeon-vllm-ultimate:latest (Spark / GB10) or ghcr.io/aeon-7/aeon-vllm-ultimate-rtx:latest (RTX discrete).

Qwen3.6-27B-AEON-Ultimate-Uncensored-Text-NVFP4-MTP

[image]

Deployment, operations & benchmarks → [github.com/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash](https://github.com/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash) The GitHub repo is the source of truth for the production deployment guide, hardware-tuned docker-compose configs, full configuration reference, measured benchmarks, and AGENTS.md — an operator's manual that pre-empts common stale-documentation traps.

🙏 Reference recipe credit: The modelopt + MTP graft pipeline used to build this variant is based on **sakamakismile**'s validated Qwen3.6-27B-NVFP4-MTP series (22K+ downloads). They worked out the modelopt config, the per-projection quantization choices, and the MTP-head graft technique on the un-abliterated base; we adapted the same recipe to AEON-Ultimate's abliterated weights. The reference benchmark numbers cited below are theirs. Full credit for the recipe → sakamakismile.

## 🆕 AEON vLLM Ultimate container (2026-06-04) `ghcr.io/aeon-7/aeon-vllm-ultimate:latest` — vLLM 0.23.0 (= :2026-06-18-v0.23.0-dflashfix) + PR #44389 NVFP4 KV cache (~3× capacity) + DFlash + TurboQuant K8V4 + AEON sm121a patches. **Same recipe family as the [`-Multimodal-NVFP4-MTP-XS`](https://huggingface.co/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Multimodal-NVFP4-MTP-XS) sibling** which has been benchmarked end-to-end (production-style greedy + nspec=15 by category: math/code peak ~45 tok/s, overall mean 34.7 tok/s; concurrent ×4 steady ~84 tok/s aggregate). This variant uses the same modelopt NVFP4 format, the same qwen3_5_mtp native head, and the same hybrid GDN+attention stack — it should serve identically with --quantization modelopt and either --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}' (native MTP) or a DFlash drafter (recommended on Spark — see container README Recipe A). The v3 image (ghcr.io/aeon-7/vllm-aeon-ultimate-dflash:qwen36-v3) remains the stable production target if you need FP8 KV + DFlash; in the new image DFlash requires --kv-cache-dtype auto (BF16). Full setup + 4-config bench comparison: container README.

Variants

What this is

This is the modelopt-format NVFP4 variant with MTP speculative decoding, text-only (vision tower stripped), of AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16 — the lossless abliteration of Qwen 3.6 27B (KL 0.000492 vs base, 0/100 refusals, multimodal preserved, hybrid GDN-aware quantization).

Specifically:

• —Body quantized to NVFP4 via nvidia-modelopt 0.43.0 with NVFP4_DEFAULT_CFG. This is the modelopt compressed-tensors format that vLLM serves through --quantization modelopt (different code path from the `-NVFP4` sibling release which uses --quantization compressed-tensors).
• —Linear-attn / GatedDeltaNet layers preserved BF16 (432 keys across 48 GDN layers). NVFP4 quantization on Mamba/SSM state collapses the recurrence; modelopt's *linear_attn.conv1d* ignore plus our explicit *linear_attn* exclude keeps these intact.
• —Vision tower stripped (333 visual keys removed, ~0.92 GB). Text-only build — no image / video input. language_model_only: true set in config.json.
• —MTP head grafted from the base Qwen/Qwen3.6-27B checkpoint (15 tensors, BF16). The base contains MTP heads but Qwen3_5ForConditionalGeneration.from_pretrained drops them during loading; the lna-lab pipeline pattern (which this build follows) explicitly grafts them back into the quantized output, giving vLLM a working drafter for --speculative-config '{"method":"qwen3_5_mtp",...}'.

Why MTP — and where it actually wins

Multi-Token Prediction (MTP) lets the model predict multiple future tokens per forward pass via the trained mtp.* head, enabling speculative decoding without a separate drafter model. The acceptance rate is high because the drafter is the model itself — same architecture, same weights, same distribution.

Measured numbers on AEON-Ultimate (this MTP family)

Reference numbers from sakamakismile's un-abliterated recipe (RTX 5090)

• —Single-stream short prompts at n=3: ~132 tok/s
• —Single-stream long-form: ~105 tok/s
• —2-parallel aggregate (256K + KV FP8): ~189–207 tok/s
• —Mean MTP acceptance length: ~3.0–4.0 (vs DFlash chains ~2.0–2.3)

The hardware-routing punchline

On RTX PRO 6000 the XS sibling beats DFlash territory (~111 tok/s vs DFlash-class ~85 we'd expect there). On DGX Spark, DFlash beats MTP by 26 % median / 52 % peak — the unified-memory bandwidth caps how much MTP's high acceptance can translate to throughput. So: MTP is a dedicated-VRAM-Blackwell variant, not a universal upgrade. Full bench data: GitHub repo Performance section.

🎯 When to pick this variant — measured hardware routing

The right speculative-decode method depends on memory architecture:

Full bench numbers: GitHub repo Performance section.

Usage

vLLM serve

# One-time: pull this repo locally
hf download AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-Text-NVFP4-MTP \
  --local-dir ./aeon-ultimate-text-nvfp4-mtp

# Serve
export VLLM_NVFP4_GEMM_BACKEND=flashinfer-cutlass
export VLLM_USE_FLASHINFER_MOE_FP4=0
export VLLM_USE_FLASHINFER_SAMPLER=1

vllm serve ./aeon-ultimate-text-nvfp4-mtp \
&
  --mamba-cache-dtype float32 \
  --trust-remote-code \
  --max-model-len 262144 \
  --max-num-seqs 32 \
  --max-num-batched-tokens 32768 \
  --gpu-memory-utilization 0.94 \
  --enable-chunked-prefill \
  --enable-prefix-caching \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --enable-auto-tool-choice \
  --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}'

num_speculative_tokens=3 is the canonical setting for qwen3_5_mtp. Higher values diverge the drafter further from the target distribution and acceptance falls.

Configuration notes

• —`--quantization modelopt` is required (not compressed-tensors — different format).
• —`--speculative-config '{"method":"qwen3_5_mtp", ...}'` activates the grafted MTP head as the spec-decode drafter. No external drafter download needed — the head is in the safetensors of this repo.
• —`--gpu-memory-utilization 0.94` is the validated cap on RTX PRO 6000; 0.95 causes the FlashInfer NVFP4 GEMM autotuner to OOM on first boot. See the GitHub repo's RTX PRO 6000 page for the same OOM behavior under DFlash.

Quantization recipe

• —Tool: nvidia-modelopt 0.43.0 with NVFP4_DEFAULT_CFG
• —Loader: Qwen3_5ForConditionalGeneration.from_pretrained (multimodal-preserved class)
• —Calibration: neuralmagic/calibration LLM split, 20 samples × 8192 tokens
• —Excluded from quantization (kept BF16):
• —lm_head, proj_out.*, *router*, *mlp.gate.* (NVFP4DEFAULTCFG)
• —*linear_attn.conv1d*, *mixer.conv1d* (NVFP4DEFAULTCFG)
• —*linear_attn* (added — full GDN preservation)
• —*visual* (added — vision tower preservation)
• —*mtp* (added — MTP head preservation)
• —*output_layer*, output.*
• —Vision strip: post-export, model.visual.* keys (333 tensors, ~0.92 GB) removed; vision_config removed from config.json; language_model_only: true set; preprocessor configs cleaned
• —MTP graft: 15 tensors copied bf16 from Qwen/Qwen3.6-27B after modelopt export (AutoModelForCausalLM.from_pretrained drops them; explicit graft restores)
• —Pipeline: lna-lab/GGUF-to-NVFP4-SM120 reference recipe, adapted for AEON-Ultimate-BF16 input + separate MTP source

Provenance & credits

• —BF16 source: `AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16`. See that card for the full abliteration pipeline.
• —MTP graft technique: lna-lab/GGUF-to-NVFP4-SM120 (docs/MTP_GRAFT_RECIPE.md)
• —Reference benchmark recipes: `sakamakismile/Qwen3.6-27B-Text-NVFP4-MTP`
• —Quantization: NVIDIA TensorRT Model Optimizer (nvidia-modelopt 0.43.0)
• —Base: Alibaba Qwen team — Qwen/Qwen3.6-27B

License + responsibility

Apache 2.0, inherited from Qwen/Qwen3.6-27B. This is an uncensored model. Read the full User Responsibility & Arbitration Clause on the BF16 source card before deploying. Summary: you implement downstream safety layers (input validation, output filtering, content moderation, audit logging, rate limiting, access controls, human-in-the-loop for high-risk workflows). The model has no opinions of its own — you supply the opinions, the judgment, and the ethics.

☕ Support the work

If this release has been useful, tips are deeply appreciated — they go directly toward more compute, more models, and more open releases.

<table align="left"> <tr><td align="left"> <strong>₿ Bitcoin (BTC)</strong><br/> <img src="https://raw.githubusercontent.com/AEON-7/AEON-7/main/assets/qr/btc.png" alt="QR" width="200"/><br/> _{<code>bc1q09xmzn00q4z3c5raene0f3pzn9d9pvawfm0py4</code>} </td></tr> <tr><td align="left"> <strong>Ξ Ethereum (ETH)</strong><br/> <img src="https://raw.githubusercontent.com/AEON-7/AEON-7/main/assets/qr/eth.png" alt="QR" width="200"/><br/> _{<code>0x1512667F6D61454ad531d2E45C0a5d1fd82D0500</code>} </td></tr> <tr><td align="left"> <strong>◎ Solana (SOL)</strong><br/> <img src="https://raw.githubusercontent.com/AEON-7/AEON-7/main/assets/qr/sol.png" alt="QR" width="200"/><br/> _{<code>DgQsjHdAnT5PNLQTNpJdpLS3tYGpVcsHQCkpoiAKsw8t</code>} </td></tr> <tr><td align="left"> <strong>ⓜ Monero (XMR)</strong><br/> <img src="https://raw.githubusercontent.com/AEON-7/AEON-7/main/assets/qr/xmr.png" alt="QR" width="200"/><br/> _{<code>836XrSKw4R76vNi3QPJ5Fa9ugcyvE2cWmKSPv3AhpTNNKvqP8v5ba9JRL4Vh7UnFNjDz3E2GXZDVVenu3rkZaNdUFhjAvgd</code>} </td></tr> </table>

Ethereum L2s (Base, Arbitrum, Optimism, Polygon, etc.) and EVM-compatible tokens can be sent to the same Ethereum address.