aleada/pack-integrity-check

Quantized pack integrity check

Quantization can drop or disable part of a model without failing. The pack loads, serves, and answers correctly — while a component its card says it preserved is either absent or present-and-ignored by the runtime. Nothing errors, and nothing in the logs says so.

Everything runs in the visitor's browser. There is no backend and no token: it reads safetensors headers and config.json over ranged requests, exactly what any visitor could read themselves. A 27 GB pack costs a few kilobytes to inspect.

What it checks

1. 1.Does every exclusion entry name a real module? An entry naming something the architecture does not have protects nothing. The common case is a vision tower: Llama and Pixtral call it vision_tower, Qwen calls it visual, and the documented default names the former.
2. 2.Did any module of the source model fail to reach the pack? from_pretrained does not materialise tensors the AutoModel class has no slot for, so an auxiliary head living inside the checkpoint never reaches the quantizer and never reaches the artifact.
3. 3.Is anything at source precision without being declared? A runtime builds those quantized, looks for a packed weight, finds a plain one, and skips it.

The first two are observations. The third is a prediction, and says so.

Two implementations, kept in step mechanically

The rules exist twice: in ../pack_check.py for the bulk scanner, and in pack_check.js here because a static Space has no Python. That is a real risk — not that the port is wrong today, but that the two drift later and only one of them is ever reviewed.

So the agreement is enforced rather than trusted:

python gen_convention_golden.py   # Python answers the shared cases
node parity.mjs                   # JavaScript must reproduce them

conventions.cases.json holds inputs only. The expectations are generated from Python, never typed by hand — a hand-written expectation records what someone believed on the day. Change a rule on either side, regenerate, and the port fails until it agrees. Verified that it can fail: breaking the regex anchoring (re.match semantics to search) is caught immediately.

Why the rules are per format

A bare mtp covers a whole head under AWQ's substring rule and covers nothing under compressed-tensors' exact-match rule. Every regex resolves through Python's re.match, which anchors at the start: re:layers.0 does not match mtp.layers.0.….

These were read out of vLLM's source, not from documentation. A format that is not on the list reports not checked rather than OK — a checker that cannot tell "healthy" from "not checked" is not a checker.

Limits

• —A clean result is not a quality measurement. This never loads the model and never runs it. A pack can pass here and still be a poor quantization.
• —Rules extracted from vLLM 0.23.1rc1.dev552+g4559c43a9 on 2026-08-11. They are not a stable public API upstream.
• —Check 3 tolerates the wrapper-level difference between the names written into an exclusion list and the names tensors are stored under, because vLLM resolves that with a per-architecture mapper this cannot replicate from metadata. It errs toward "declared" — missing some real cases rather than accusing correct packs.
• —Gated and private repos cannot be read.

Deploying

HF_TOKEN=... ./deploy.sh aleada/pack-integrity-check

Static Spaces are free; Gradio and Docker Spaces require PRO. A Gradio version of the same checker lives in `../space/` for if that changes — it shares pack_check.py directly and needs no port.