Modelpublic

jan1k/Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-GGUF

sourceHugging Faceapache-2.0updated 3d agoView on Hugging Face

3likes1.4kdownloads

Model Card

Qwen3.6-35B-A3B-Uncensored-Genesis — NVFP4 GGUF

NVFP4 quantisation of LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Final-GGUF, built with advanced-gguf-quantizer (a llama.cpp fork focused on NVFP4/MXFP6 quantization).

This is the non-Hermes version — Genesis tensor repair on the HauhauCS uncensored base, without the Hermes finetune transfer. For the Hermes version, see jan1k/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final-NVFP4-GGUF.

Files

v4 — Recommended (inline scales, LM Studio compatible)

File	Size	Tensors
`Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-v4.gguf`	~20 GB	733
`mmproj-Qwen3.6-35B-A3B-Uncensored-Genesis-Final-F16.gguf`	~899 MB	vision projector

Re-quantised 2026-09-24 from Luffy's updated Q8KP source.

v4 uses --nvfp4-inline-scales-only to emit native inline UE4M3 scales without separate .scale/.input_scale tensors. This is required for LM Studio / Pelican and other runtimes that do not support the extended NVFP4 scale tensor contract.

Imatrix variants will follow after the plain v4 is confirmed working in target runtimes.

Source


Source GGUF	`Qwen3.6-35B-A3B-Uncensored-Genesis-Final-Q8_K_P.gguf` (43.6 GB, 10.06 BPW)
Architecture	`qwen35moe` (MoE), 35B total / ~3B active, 40 layers
Context	262144
`general.file_type`	39 (`LLAMA_FTYPE_MOSTLY_NVFP4`)
MTP/NextN	none

Two-step pipeline: Q8KP -> F16 intermediate -> NVFP4. The F16 step gives the NVFP4 encoder clean data (no Q8_0 quantization noise).

Tensor mix (v4)

type	count	notes
F32	331	norms, ssm scalars, gate inputs
F16	175	sensitive weights (blk.0 attn, ssm, etc.)
Q6_K	1	`output.weight`
NVFP4	226	bulk weights
total	733
separate `.scale`/`.input_scale`	0	inline UE4M3 only

Format fix (v3/v2 -> v4)

The older v2/v3 files use the extended NVFP4 format with separate .scale and .input_scale tensors. Runtimes like LM Studio / Pelican only read inline UE4M3 scales and ignore the separate scale tensors, producing garbage output.

v4 uses --nvfp4-inline-scales-only to:

—Skip writing .scale and .input_scale tensors
—Force tensor_scale=1.0 (no pre-scaling)
—Force input_scale=identity
—Make inline UE4M3 scales self-contained

Tensor protection policy

F16 singular-collapse protection:

tensor	type
`blk.0.attn_gate.weight`	F16
`blk.0.attn_qkv.weight`	F16
`blk.0.ffn_down_exps.weight`	F16
`blk.13.ffn_down_exps.weight`	F16

F32 architecture-specific protection:

tensor	reason
`blk.*.attn_norm.weight`	1D norm
`blk.*.post_attention_norm.weight`	1D norm
`blk.*.attn_q_norm.weight`	1D norm
`blk.*.attn_k_norm.weight`	1D norm
`blk.*.ssm_norm.weight`	1D norm
`output_norm.weight`	1D norm
`blk.*.ssm_conv1d.weight`	CUDA SSM conv kernel requires F32
`blk.*.ssm_dt.bias`	SSM scalar, kernel compatibility
`blk.*.ssm_a`	SSM scalar, kernel compatibility
`blk.*.ffn_gate_inp_shexp.weight`	shared expert gate input
`token_embd.weight`	F16 (embedding quality)

Forced NVFP4 (do not push lower):

tensor	type
`blk.0.ssm_out.weight`	NVFP4
`blk.1.attn_gate.weight`	NVFP4
`blk.1.attn_qkv.weight`	NVFP4

Usage

bash

llama-cli -m Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-v4.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-Uncensored-Genesis-Final-F16.gguf \
  --jinja -c 131072 -ngl 99

—Set K cache and V cache quantization to F16
—Set GPU offload to maximum, active experts to 8
—Set number of layers for which to force MoE weights onto CPU to 40

Hardware

—Blackwell (RTX 50xx): native FP4 path, fastest
—Ampere (RTX 30xx): NVFP4 inference works via fallback kernels
—Quantisation was done CPU-only (Ampere CUDA NVFP4 encoder hangs on MoE)

Reproducibility

bash

# Step 1: Q8_K_P -> F16 intermediate
llama-quantize --allow-requantize \
  Qwen3.6-35B-A3B-Uncensored-Genesis-Q8_K_P.gguf temp_f16.gguf F16 6

# Step 2: F16 -> NVFP4 v4 (inline scales only, CPU-only)
CUDA_VISIBLE_DEVICES="" llama-quantize \
  --allow-requantize \
  --nvfp4-inline-scales-only \
  --tensor-type-file tensor_types_protection.txt \
  temp_f16.gguf \
  Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-v4.gguf \
  NVFP4 6

# Cleanup
rm temp_f16.gguf

Credits

—Base model: HauhauCS (0/465 refusals)
—Genesis tensor repair: LuffyTheFox
—NVFP4 quantisation: jan1k
—Quantiser: advanced-gguf-quantizer