edougawa/ThinkingCap-Qwen3.6-27B-Abliterix-NVFP4

ThinkingCap Qwen3.6 27B Abliterix — NVFP4

NVIDIA ModelOpt NVFP4 unified Hugging Face checkpoint produced from the ThinkingCap Qwen3.6-derived Abliterix checkpoint. The checkpoint declares the Qwen3_5ForConditionalGeneration architecture and is intended for vLLM on Blackwell-class NVIDIA hardware.

This is an abliterated model. Treat its safety behavior and outputs as altered; evaluate it for your own deployment.

Quantization

• —ModelOpt: 0.45.0
• —PyTorch / CUDA: 2.11.0 / CUDA 13.0
• —Transformers: 5.9.0
• —Format: ModelOpt NVFP4, group size 16, activation-aware clipping calibration
• —Calibration: 256 instruction conversations from HuggingFaceH4/ultrachat_200k and 128 image-question examples from the AI2D subset of HuggingFaceM4/the_cauldron
• —Text calibration length: 2,048 tokens; calibration images are resized to at most 1,500,000 pixels without truncating multimodal placeholder tokens
• —KV cache: BF16/default (kv_cache_quant_algo: null)
• —Execution hardware: two NVIDIA B200 GPUs; intended runtime: vLLM on Blackwell hardware, including DGX Spark / GB10-class deployments

Features

• —Text, image and video processor assets are included.
• —The vision/video tower and merger remain BF16, as do embeddings, normalizations, LM head, hybrid linear-attention modules, and MTP-sensitive paths.
• —The source configuration does not contain an audio tower or audio weights; audio capability is therefore not claimed.
• —The underlying Qwen MTP configuration is preserved. Enable self-speculation only when supported by your installed vLLM build.

No benchmark or quality claims are made here; evaluate the quantized checkpoint on your own representative text and multimodal workloads.

vLLM

vllm serve edougawa/ThinkingCap-Qwen3.6-27B-Abliterix-NVFP4 \
  --quantization modelopt_fp4 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

For a non-speculative launch, omit --speculative-config.