tiyuvta/GLM-5.3-Flash-NVFP4-B200-hybrid

GLM-5.3-Flash NVFP4 B200 hybrid (Puzzle Table 2)

Hybrid-precision mint of zai-org/GLM-5.3-Flash for NVIDIA B200 / sm_100, not for the RTX PRO 6000 pin. Recipe follows NVIDIA Nemotron-Labs-3-Puzzle-75B-A9B Table 2 (NVFP4 column): per-operator hybrid, not a uniform 4-bit checkpoint.

Built with the tiyuvta serving engine, by tiyuvta (inference.tiyuvta.ai).

Run GLM-5.3-Flash today through an API. Open GLM-5.3-Flash on tiyuvta and use model id zai/glm-5.3-flash. That endpoint serves the PRO 6000 mint (tiyuvta/GLM-5.3-Flash-NVFP4 @ 2bade47d). This B200 hybrid is a different numeric program and a different layout. It is not a candidate for that pin. Speculative decoding on the live endpoint is drafted by DFlash 2 (inco.ai). Get an API key and send the first request.

• —189.1 GB across 19 safetensors shards. 38,770 source tensors classified fail-closed: 37,290 NVFP4, 204 FP8, 86 FP32, 1,190 BF16.
• —Stored tensors: 113,554 = 3 x 37,290 NVFP4 triples + 2 x 204 FP8 pairs + 86 FP32 + 1,190 BF16.
• —Source is the vendor's BF16 twin @ f12e0fe1f6b2ea274c11a569582edfd99d993c5e, not their FP8 release.
• —PRO 6000 Four-Over-Six sibling (same keep-list as the live pin, modelopt scale layout): tiyuvta/GLM-5.3-Flash-NVFP4-4o6.
• —Upstream technical report: GLM-5: from Vibe Coding to Agentic Engineering (arXiv 2602.15763).

How it was made

try_tensorrt=True is a no-op on the mint GPU (sm89). The Swizzle32x4x4 plane is written explicitly. That is the opposite of the 4/6 remint, which keeps `trytensorrt=False` so the serving engine's existing modelopt-layout reader can eat it. This file is not a drop-in for that reader. B200 tensor-core and a dp4a indexer that addresses K-major / swizzled scales are the consumers.

LAYOUT.json in this repo is the operator map, not a comment.

Precision split (Puzzle Table 2 -> GLM-5.3-Flash)

KDA o_proj stays BF16: it is not in the kda6 six. Low-rank f_b / g_b stay BF16.

Versus the live pin / the 4/6 remint, in one list:

1. 1.Experts stay NVFP4, but calib, pack, and scale layout change (W4A4-shaped, max-calib, K-major, swizzled).
2. 2.KDA six: BF16 keep -> FP8.
3. 3.MLA QKV/o: NVFP4 -> BF16 keep.
4. 4.Router: BF16 -> FP32.
5. 5.input_scale for expert W4A4 tensor-core is not in this file yet.

The gate this has passed (and what it has not)

Census on the mint: 37,290 NVFP4 / 204 FP8 / 86 FP32 / 1,190 BF16, every source tensor classified as exactly one bucket or the mint aborted. KDA layers asserted as exactly those six names and shapes.

Not yet a serving pin. Kernel-check, argmax-margin, and vendor-default sampled decode on sm_100 are still owed on these exact bytes. Do not load this in place of 2bade47d on PRO 6000. Do not treat weight mint as W4A4 serving: the activation scale plane is missing.

FR-Spec ranks (the .txt files)

Same three traffic classes as the q38 and ornith cards. Tokenizer is unmodified from zai-org/GLM-5.3-Flash (tokenizer.json sha256 19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d). These ranks are a vocab artifact of that tokenizer, minted 2026-08-30: plain text, one token id per line, most-frequent first, 32,768 lines. They drive the serving engine's load-time trim on a safetensors trunk. Verification still uses the full vocabulary, so a trim cannot change greedy output.

A per-requant own-generation remint of ranks from this checkpoint is still owed after the B200 serving gate.

Running it

This is a hybrid glm5_next checkpoint with a CUTLASS-swizzled NVFP4 scale plane. A stock transformers load is not the path. The live PRO 6000 repack path on the serving engine is not the path either.

Only a reader that indexes K-inner packed weights and Swizzle32x4x4 scales will do the right matmul. Native chat dialect is the checkpoint's own chat_template.jinja ([gMASK]<sop>, <|user|> / <|assistant|>, GLM tool calls). Do not pattern-match ChatML. Honour all three eos ids in generation_config.json.

Files

The vision tower is present in the source architecture and is not exercised by this mint's census.

Attribution and licence

MIT, following upstream zai-org/GLM-5.3-Flash. All model capability belongs to the GLM-5 team; this repository contributes a hybrid quantization. Speculative decoding on the live tiyuvta endpoint is drafted by DFlash 2 (inco.ai), used with the authors' written permission.

@misc{glm5team2026glm5vibecodingagentic,
      title={GLM-5: from Vibe Coding to Agentic Engineering},
      author={GLM-5-Team},
      year={2026},
      eprint={2602.15763},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2602.15763},
}