Modelpublic

DmitryDB/Qwen3.6-35B-A3B-Heretic-MOE-v3

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes903downloads

Model Card

Qwen3.6-35B-A3B Heretic-MoE-v3

This repository contains the reproducible Heretic-MoE-v3 release derived from `Qwen/Qwen3.6-35B-A3B`. The search and release tooling is developed in `dborzoff/Heretic-MOE`.

Release status: complete. Published weight files are single-file artifacts. Reproducibility manifests record sizes and SHA-256 digests; private evaluation prompts and generated response archives are not published.

Release variants

Variant	Rechecked trial	SRG	R-side	Keywords	PPL drift
Balanced	T402	-0.01928	33.8%	1/136	0.03%
Max	T541	-0.02155	30.9%	0/136	0.21%

Balanced and Max are selected only after a high-fidelity recheck. If both roles resolve to the same physical checkpoint, the manifest records the alias instead of duplicating weights.

—Balanced is the default choice when preserving the upstream model is the priority.
—Max is the stronger refusal-removal candidate within the same 0.5% PPL preservation gate.

Search and verification contract

Stage	Contract
Exploration	120 alternating Random/Scrambled-Sobol trials
Refinement	Shared multivariate TPE study to 600 completed trials on two resident GPU workers
Search metrics	Sparse refusal geometry, refusal-side rate, keyword flags, and absolute perplexity drift
Preservation gate	Maximum absolute PPL drift: 0.5%
Finalist set	Diverse TOP-6, not six near-duplicates from one scalar ranking
Recheck	64 windows x 1024 tokens
Final roles	Pareto-valid Balanced and Max

The search stores numeric metrics, parameters, manifests, and hashes under `research/`. Private evaluation prompts and generated response archives are not published.

Published artifacts

Family	Balanced	Max
Single-file Safetensors	BF16, INT8 Lean ConvRot, `NVFP4_W4A16`, `NVFP4_W4A4`	BF16, INT8 Lean ConvRot, `NVFP4_W4A16`, `NVFP4_W4A4`
GGUF	F16, Q80, Q6K, Q4KM, IQ4XS, IQ3M, IQ2_XXS	F16, Q80, Q6K, Q4KM, IQ4XS, IQ3M, IQ2_XXS
Importance matrix	`balanced.imatrix`	`max.imatrix`

The repository also contains one shared BF16 multimodal projector. Vision weights are unchanged by the Heretic-MOE text-model transformation, so the projector is not duplicated for Balanced and Max.

Repository layout

Path	Contents
`balanced/safetensors/`	Single-file Balanced BF16, INT8 Lean ConvRot, and NVFP4 weights
`max/safetensors/`	Single-file Max BF16, INT8 Lean ConvRot, and NVFP4 weights
`balanced/gguf/`	Balanced F16 master and six derived GGUF quants
`max/gguf/`	Max F16 master and six derived GGUF quants
`balanced/imatrix/`	Importance matrix used for Balanced Q6/Q4/IQ quantization
`max/imatrix/`	Importance matrix used for Max Q6/Q4/IQ quantization
`shared/vision/`	Unmodified shared BF16 multimodal projector for GGUF runtimes
`research/`	Search, recheck, export, quantization, integrity, and release manifests

Q6, Q4, and IQ files are generated from the matching winner-specific imatrix. The ordinary layers of each IQ2 file use IQ2XXS. Eleven MTP weight tensors in `blk.40` use Q4K because the inference calibration path cannot activate the MTP prediction head and therefore cannot provide imatrix entries for it. This exception is recorded in the release manifests rather than silently producing an invalid low-bit file.

NVFP4 profiles

Profile	Quantization policy	Intended trade-off
`NVFP4_W4A16`	MSE-calibrated NVFP4 weights for selected MoE/MLP projections, FP8 attention and KV handling, higher-precision MLP activations	Quality-first NVFP4 option
`NVFP4_W4A4`	The same selected weight/module map plus dynamic NVFP4 MLP and LM-head input activations; FP8 attention and KV handling remain unchanged	Lower activation memory and faster Blackwell-oriented execution

The W4A4 tensor and quantizer schema is matched against `nvidia/Qwen3.6-35B-A3B-NVFP4`: 291 quantized modules and 331 enabled input quantizers. The Heretic-MOE values are independently calibrated from the Balanced and Max BF16 masters, so they are not byte copies of NVIDIA's upstream weights.

The multimodal projector remains a separate artifact for GGUF runtimes. No W8/W4 Lean ConvRot file is claimed for this release because no verified Qwen3.6-MoE consumer recipe was available for that layout. INT8 Lean ConvRot and both NVFP4 profiles are specialized packed layouts and require a compatible loader; they are not drop-in replacements for the BF16 Transformers checkpoint.

Compatibility boundary

Qwen3.6-35B-A3B is a native multimodal MoE model. GGUF model and projector compatibility is verified with a pinned official llama.cpp commit. A compact safetensors file is not described as a working text encoder until its target consumer loads it and passes a conditioning smoke test.

Reproducibility

The release includes:

—the immutable search and finalist manifests;
—numeric, text-free candidate and recheck summaries;
—source, toolchain, calibration-corpus, and output SHA-256 hashes;
—exact Heretic-MOE and llama.cpp commits;
—conversion and quantization manifests;
—validation results for every published artifact.

Heretic-MOE source and reproducibility tools: <https://github.com/dborzoff/Heretic-MOE>

License

The derivative follows the upstream Apache-2.0 license. Review the upstream Qwen3.6 model card and license before deployment.