Modelpublic

wrldsuksgo2mars/GLM-5.2-EXL3-K3-calibrated-v1

sourceHugging Facemitupdated 1mo agoView on Hugging Face

0likes70downloads

Model Card

GLM-5.2 EXL3 K3 calibrated v1

This repository contains a calibrated EXL3 K3 variant of `zai-org/GLM-5.2` for GLMRT. The three dense decoder layers, attention, routers, shared experts, embeddings, head, and native MTP layer retain their source tensors. Only the 256 routed experts in base decoder layers 3 through 77 are replaced by EXL3 K3 MCG trellis/suh/svh/mcg tensors.

The artifact uses a single resident expert representation on each of four Spark TP ranks. Every rank retains its 512-wide intermediate slice of every expert; the runtime does not keep native and EXL3 copies resident together.

Quantization

—Source: zai-org/GLM-5.2, revision b4734de4facf877f85769a911abafc5283eab3d9
—Format: EXL3, 3 physical trellis bits per routed-expert weight
—Codebook: MCG
—Output-scale search: automatic, folded into the stored rotations
—Calibration: 1,080,625 GLM-5.2 tokenizer tokens spanning general text, English and Chinese, code/agentic work, mathematics/reasoning termination, and structured output
—Sparse coverage: natural top-8 routes, deterministic adjacent-router recovery for deficient experts, then an explicitly recorded isotropic Hessian residual only for any remaining deficit
—Quantizer: the content-pinned GLMRT GPTQModel fork

The calibration, held-out, and screening splits are source-disjoint. The complete immutable plan, projection evidence, retained-native proof, and artifact manifest are maintained by the GLMRT project rather than shipped as large private recovery files in this standard model repository.

Runtime

The artifact is intended for GLMRT's generated SparkInfer SM121 EXL3 K3 TP4 path. Generic Transformers metadata is included, but compatibility with other EXL3 runtimes has not been claimed.

The qualification and performance measurements below were made with the GLMRT v6 inference engine.

Qualification

The exact artifact passed the content-bound GLMRT qualification gate. Both arms used identical coordinator and Spark binaries, the balanced profile, dSpark speculation, and a 400 W coordinator power limit.

Structural and quantizer evidence

—Routed projections quantized: 57,600
—EXL3 tensors: 230,400
—Retained native tensors byte-compared: 1,985
—EXL3 tensor payload: 254.00 GiB
—TP4 resident payload per Spark: 64.00 GiB
—Aggregate Hessian-weighted relative projection error: 0.0063075206
—End-to-end serving decision: accepted

Decode and acceptance

Workload	NVFP4	EXL3	Ratio
Weighted decode	27.016 tok/s	28.679 tok/s	1.062x
Orchid repeat	64.309 tok/s	71.439 tok/s	1.111x
dSpark accepted drafts	76.61%	72.65%	0.948x

—Policy: explicit decode-optimized tradeoff; the ordinary 0.950x acceptance and per-cell prefill floors were not used
—Selected acceptance floor: 0.940x of NVFP4
—Candidate semantic contracts: 35/35 passed

Prefill

Context	Prefill rows	NVFP4 tok/s	EXL3 tok/s	Ratio
0	1,024	590.0	700.7	1.188x
0	2,048	977.2	1,031.1	1.055x
0	4,096	1,488.8	1,345.8	0.904x
0	8,192	1,761.9	1,607.0	0.912x
0	16,384	1,803.6	1,683.6	0.933x
0	32,768	1,773.3	1,709.7	0.964x
32,768	1,024	850.3	676.7	0.796x
32,768	2,048	965.0	1,020.3	1.057x
32,768	4,096	1,483.6	1,374.9	0.927x
32,768	8,192	1,717.5	1,645.8	0.958x
32,768	16,384	1,714.2	1,692.8	0.988x
32,768	32,768	1,684.1	1,684.9	1.000x
65,536	1,024	768.2	663.7	0.864x
65,536	2,048	960.3	1,015.0	1.057x
65,536	4,096	1,480.1	1,349.3	0.912x
65,536	8,192	1,646.8	1,619.9	0.984x
65,536	16,384	1,597.3	1,649.1	1.032x
65,536	32,768	1,571.9	1,622.4	1.032x
131,072	1,024	827.1	658.4	0.796x
131,072	2,048	956.8	1,010.0	1.056x
131,072	4,096	1,362.0	1,298.3	0.953x
131,072	8,192	1,414.9	1,514.4	1.070x
131,072	16,384	1,518.8	1,523.2	1.003x
131,072	32,768	1,484.3	1,498.6	1.010x
262,144	1,024	793.3	808.9	1.020x
262,144	2,048	904.0	969.0	1.072x
262,144	4,096	1,221.2	1,179.6	0.966x
262,144	8,192	1,317.5	1,310.8	0.995x
262,144	16,384	1,308.4	1,324.4	1.012x
262,144	32,768	1,290.8	1,306.8	1.012x

Minimum prefill cell ratio: 0.796x. Selected per-cell prefill floor: 0.790x of NVFP4.

Tool use and startup

—Tool-call score: 125/138 points (NVFP4: 120/138; 1.042x)
—Expert resident preload: 17,212.3 ms (NVFP4: 24,551.2 ms; 0.701x)
—Full expert service handoff: 17,712.4 ms (NVFP4: 24,975.8 ms; 0.709x)
—Native EXL3 parity: TP ranks 0, 1, 2, 3; calibrated layer 3; rows 1, 3, 9, 10, 129, 257, 513, 1025, 2049, 2064

Reproducibility identity

—GLMRT WIP engine: wip-exl3-fp32-priority-e9d90fb41fcf-9f564406a95d
—SparkInfer revision: f78061114b5575a2969e35b248ed6397424a306c
—Coordinator slot SHA-256: e9d90fb41fcfb73b5c90991549458c020f38b7acb8c7dc717edfbc53c1cb86f0
—Spark slot SHA-256: 9f564406a95db11707ec9dc7ddfe83079aa0b31be4845a84f2cd7b6b7dac84be
—Quantizer execution upgrade SHA-256: 9a1843a7cc13977a21b53ceb9f57bdf938df826d3b4347507946aa186e39f5e1
—Qualification evidence SHA-256: 61dc821627b22cb9bc469c8db54bf5d5256787ac46296efda092f71172161c74
—Hub revision: assigned and verified immediately after the initial upload

Before publication this marker is replaced mechanically from the signed final structural, quantizer, and serving reports. The publication builder rejects a card that retains the marker or does not cite the exact serving-report hash.

License and attribution

This derivative follows the source model's MIT license. See LICENSE and the original `GLM-5.2` model card for upstream architecture, intended-use, and limitation details.