Modelpublic

philipjohnbasile/GLM-5.2-Demolition-q4a4-soul-MLX

sourceHugging Facemitupdated 16d agoView on Hugging Face

12likes1.6kdownloads

Model Card

GLM-5.2-Demolition · q4a4-soul (v3)

Explore the model guide · All public work

Release at a glance

	This artifact
Purpose	An expert-pruned GLM 5.2 build with quantized root weights and separately published adapter experiments.
Runtime	MLX with the release's GLM DSA setup. This is distinct from the unpruned iliria SSD container.
Status	Recorded runtime; see the evidence and limits below.
Tensor download	104.89 GB (97.68 GiB) of root `.safetensors` files, including any root sidecars. This is a file-size total, not peak RAM.
Read first	Root tensors are about 104.89 GB (97.68 GiB). A whole-repository download includes many adapter checkpoints and is substantially larger.

A demolition of `zai-org/GLM-5.2` (744B total / 39B active MoE, MIT) down to a ~105 GB (98 GiB) 4-bit model that loads and runs fully on a single Apple M5 Max (128 GB). v3's distinguishing move is soul-targeted expert pruning — the kept experts are chosen by saliency measured on our facet data, not a generic corpus — plus a deliberately pure vanilla-code core with swappable heritage "souls" mounted on demand.

Download scope

The root model tensors total 104.89 GB (97.68 GiB). This repository also contains nested adapter checkpoints: all .safetensors files together total 187.51 GB. An unfiltered download retrieves those additional experiments too. This does not change the core model's memory use. Select core files first, then download a specific adapter if your recipe needs it:

bash

hf download philipjohnbasile/GLM-5.2-Demolition-q4a4-soul-MLX \
  --include 'model*.safetensors' --include 'model.safetensors.index.json' \
  --include 'config.json' --include 'tokenizer*' --include 'chat_template.jinja' --include '*.py' --include '*.md' \
  --local-dir ./glm52-demolition-core

This is a download command, not a validated launch command. Read the bundled GLM DSA setup and extended model card before running it.

Architecture: a PURE core + swappable souls

—CORE (always-on): just the vanilla languages, done excellently — Python · TypeScript · JavaScript · Rust · Go · HTML · CSS · SQL · Postgres. No frameworks, no baked specialties. Latest versions, vanilla stdlib.
—SOULS (mount per-request — the model factory): small LoRA adapters that name a field's masters to activate latent eliteness:
—art (Basquiat/Haring/Banksy/Sol LeWitt/Casey Reas) · music (Bach/J Dilla/Eno/WALL-E) · design (Rams/Bauhaus) · perfumery (Beaux/Guerlain/Ellena) · science (Feynman/Darwin/Sagan) · legacy (K&R/Knuth/Dijkstra/Hopper) · security (Saltzer-Schroeder/Aleph-One, purple-team) · gamedev (Carmack/Handmade-Hero, vanilla from-scratch) · fullstack (htmx/Go-stdlib/Postgres) · math · dataviz · prose · architecture · research.

Running the full 744B instead

This repo is a pruned ~105 GB (98 GiB) MLX build; its weights are not compatible with iliria.

If you want the full 744B GLM-5.2 (unpruned, int4) on the same 128 GB Mac, **iliria** streams the full model's experts from SSD. A ready-to-run container is at GLM-5.2-colibri-int4-with-int8-mtp — ~384 GB, no conversion needed.

The demolition lineage (honest)

ver	prune	quant	size	result
v1	keep 30% experts (generic calib)	3-bit	99 GB	broke — hallucinates, sentence-loops
v2	keep 23% experts (code calib)	4-bit	98 GB	design coherent; trivia gone (by design)
v3	keep 23% experts (soul calib)	4-bit	104.89 GB / 97.68 GiB	coherent FOCUS-9 vanilla code (healed)

Why 4-bit, not 3: 3-bit was just below the quality cliff; 4-bit is just above it and MLX's best-optimized kernel (cleanest packing). 2-bit is worse. These experiments did not establish superiority over an appropriately sized unpruned model; this artifact is one measured pruning-and-quantization experiment; it does not establish an optimal recipe.

Method

1.Saliency (23_stream_calibrate) on our facet corpus → score each routed expert.
2.Prune (24_apply_prune --ratio 0.77) → keep the top-saliency experts.
3.Re-quantize (24b_stream_requantize --bits 4) → uniform 4-bit experts, 4-bit attn, 6-bit head.
4.Heal (06_heal_lora) — LoRA on vanilla FOCUS-9 gold; souls heal separately per facet.

Honest scope

—Speed: ~11–14 tok/s single-stream — memory-bandwidth-bound (recorded on M5 Max; an earlier version of this line said ~10; the recorded speculative-decoding gain was only ~1.05× here, so it's not used).
—Strengths: the FOCUS-9 vanilla languages + whichever soul is mounted. Not general trivia — those experts were deliberately pruned. Best driven by a verifier-first agent (the compiler steers each line).
—Eval: HumanEval-164 pass@1 = 114/164 (69.5%) — full set, single-shot, scored on hidden tests by real verifiers (the easy n=20 subset was 95%). This is a result on the stated single-shot protocol, not a controlled comparison with GPT-4 or current frontier models. An agent loop was not quantified by this single-shot score. Strong on writing vanilla FOCUS-9 functions from a spec; weaker on hard debugging/multi-step and off-distribution prompts. Honest scope: this is a pruned research artifact. The published extended model card contains additional project context; MISSION_SUMMARY.md is not included in this Hub revision.

Built with the open pipeline at glm52-demolition. Public (MIT — GLM-5.2 is Z.ai Pure-Open).