CoolFace
Modelpublic

jhone888/Ornith-1.5-35B-A3B-FULL-OBLITERATED-NVFP4

sourceHugging Facemitupdated 28d agoView on Hugging Face
2likes1kdownloads
Model Card

Ornith-1.5-35B-A3B-FULL-OBLITERATED — NVFP4 (full: MTP + vision)

NVFP4 quantization of jhone888/Ornith-1.5-35B-A3B-FULLY-OBLITERATED — the second-pass obliterated v2, now a structurally complete checkpoint: the language model, the native MTP head (self-drafting speculative decoding, 785 tensors) and the vision tower (333 tensors), transplanted byte-identical from the v1 base.

⚠️ Uncensored model. You are responsible for its use.

Quantization (official-style recipe)

ComponentFormat
Routed experts (un-fused, 256/layer)W4A16_NVFP4, group-16
Shared experts, lm_headW4A16_NVFP4, group-16
Attention / linear-attention projectionsFP8 (per-tensor weight scale)
MTP head, vision tower, norms, embeddings, convBF16 (untouched)
KV cacheFP8 (config flag)

23.4 GB across 5 shards. Tensor-level audit against the official ornith-ai/Ornith-1.5-35B-A3B-NVFP4 recipe: weight_scale_2 = 30,841 and 291 quantized layers (FP8 130 + W4A16 161) — exact match. FP8 layers use static weight scales (no activation input_scale); verified working with the FreeToken FTW pipeline.

Note: using this checkpoint with FreeToken (FTW)

This repo ships the full multimodal config (model_type: qwen3_5_moe, nested text_config/vision_config) because the checkpoint genuinely contains the MTP head and vision tower. vLLM and transformers load it as-is.

FreeToken's `ft checkpoint` (FTW conversion) needs one change: swap config.json for the flat text-only variant, or the conversion fails with Missing MoE expert source layers. Why: with the multimodal config, FreeToken's model adapter dispatches expert loading to its BF16-bank provider, which searches for fused expert banks (model.layers.N.mlp.experts.gate_up_proj) that don't exist in any NVFP4 build (experts are stored un-fused and quantized per-expert). The flat text-only config (model_type: qwen3_5_moe_text, all text fields at top level) dispatches to the NVFP4 provider, which reads the un-fused quantized layout correctly.

The drop-in flat config is pottokao's `config.json` with one mandatory edit: architectures must be ["Qwen3_5MoeForConditionalGeneration"] (FreeToken's registry rejects ...ForCausalLM). The MTP/vision tensors are then ignored by FreeToken's text-only FTW; keep this folder's config for vLLM and other engines.

Credits