jan1k/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final-NVFP4

Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final — NVFP4 GGUF

NVFP4 quantisation of LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final-GGUF, built with advanced-gguf-quantizer (a llama.cpp fork focused on NVFP4 quantization).

• —Original Model & Genesis Tensor Repair: LuffyTheFox
• —Base Model: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive (0/465 refusals)
• —Hermes Finetune: DJLougen/hermes-qwen3.5-35b-a3b-GGUF (transferred Hermes data onto the uncensored base)
• —Architecture: Mixture of Experts (MoE) — 35B total / ~3B active per token (8 routed + 1 shared)
• —Multimodal (Vision): Supported via mmproj-Hermes3.6-35B-A3B-Uncensored-Genesis.gguf

Files

v4 — Recommended (inline scales, LM Studio compatible)

v4 uses --nvfp4-inline-scales-only to emit native inline UE4M3 scales without separate .scale/.input_scale tensors. Required for LM Studio / Pelican and other runtimes that do not support the extended NVFP4 scale tensor contract.

Imatrix variants will follow after the plain v4 is confirmed working in target runtimes.

Source

Two-step pipeline: Q8KP → F16 intermediate → NVFP4. The F16 step gives the NVFP4 encoder clean data (no Q8 quantization noise). Quantised CPU-only (Ampere CUDA NVFP4 encoder hangs on MoE tensors).

Tensor mix (v4)

Tensor protection policy

F16 singular-collapse protection:

F32 architecture-specific protection:

• —all norm weights (attn/post_attention/q/k/ssm norms)
• —blk.*.ssm_conv1d.weight, blk.*.ssm_dt.bias, blk.*.ssm_a
• —blk.*.ffn_gate_inp_shexp.weight (shared expert gate input)
• —token_embd.weight → F16

Forced NVFP4 (do not push lower):

Usage

llama-cli -m Hermes3.6-35B-A3B-Uncensored-Genesis-Final-NVFP4-v4.gguf \
  --mmproj mmproj-Hermes3.6-35B-A3B-Uncensored-Genesis.gguf \
  --jinja -c 131072 -ngl 99

• —Set K cache and V cache quantization to F16
• —GPU offload maximum, active experts 8
• —Use chat_template.jinja with --jinja; system prompts included in-repo

Hardware

• —Blackwell (RTX 50xx): native FP4 path, fastest
• —Ampere (RTX 30xx): NVFP4 inference works via fallback kernels

Reproducibility

# Step 1: Q8_K_P -> F16 intermediate
llama-quantize --allow-requantize \
  Hermes3.6-35B-A3B-Uncensored-Genesis-Final-Q8_K_P.gguf temp_f16.gguf F16 6

# Step 2: F16 -> NVFP4 v4 (inline scales only, CPU-only)
CUDA_VISIBLE_DEVICES="" llama-quantize \
  --allow-requantize \
  --nvfp4-inline-scales-only \
  --tensor-type-file tensor_types_protection.txt \
  temp_f16.gguf \
  Hermes3.6-35B-A3B-Uncensored-Genesis-Final-NVFP4-v4.gguf \
  NVFP4 6

Credits

• —Genesis algorithm & repair: LuffyTheFox
• —Base model: HauhauCS
• —Hermes dataset: NousResearch / DJLougen
• —NVFP4 quantisation: jan1k
• —Quantiser: advanced-gguf-quantizer