CoolFace
Modelpublic

nDimensional/Qwen3.5-35B-A3B-Uncensored-FP8_BLOCK

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face
1likes72downloads
Model Card

Qwen3.5-35B-A3B Uncensored (FP8_BLOCK)

A safetensors conversion and quantization of HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive (GGUF).

Model Details

ArchitectureQwen3.5 MoE hybrid attention (30 GDN + 10 full standard attention layers)
Parameters35B-A3B
Base modelQwen/Qwen3.5-35B-A3B
Source GGUFHauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive
FormatBF16 (Mixed/Compressed)
QuantizationFP8_BLOCK applied to Linear transformer layers.
Stripped layersMulti-Token Prediction (MTP) due to original HF -> GGUF conversion.
Conversion typeLossless GGUF to safetensors conversion + merge with base model vision layers + Block-wise quantization
Unquantized weights<i>Coming Soon</i>

Conversion Details

Converted using <i>coming soon</i>, which reverses transforms applied during HF -> GGUF conversion.

The vision encoder weights are copied directly from the official Qwen/Qwen3.5-35B-A3B base model, after confirming the vision encoder (mmproj) was not modified in the source GGUF.

Next, the linear weights of the transformer blocks were quantized to F8_E4M3 using llm-compressor.

Test Inference Details

  • —1x A100 (80GB)
  • —Python 3.12
  • —vllm & transformers version:
  • —transformers 5.5.0
  • —vllm nightly (latest commit tested: 7b743ba)
  • —vLLM online serve flags:
  • —--quantization compressed-tensors
  • —--max-model-len 16384
  • —--gpu-memory-utilization 0.9140 with VLLM_MEMORY_PROFILER_ESTIMATE_CUDAGRAPHS=1 environmental variable
  • —--limit-mm-per-prompt.image 4
  • —--enable-prefix-caching
  • —--enable-expert-parallel
  • —--reasoning-parser qwen3
  • —--default-chat-template-kwargs {"enable_thinking": false} disabled thinking/reasoning for vllm>=0.18.1
  • —Note: Used for batch image captioning tests.

Credits