malaiwah/qwen38-27b-fidelity-root-v1

Qwen3.8-27B BF16 root fidelity dataset (hidden form)

A root capture of Qwen/Qwen3.8-27B at revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 — 18 shards, no quantization_config, a genuinely unquantized reference — over the sealed suite-v5 shard-0 token panel (512 contexts x 2048 tokens = 1,048,064 scored positions).

What this is for

Quantization fidelity is usually reported as a KL divergence against a teacher. If the teacher was captured on a different stack than the candidate, that number contains an engine term as well as a quantization term, and the quantization part can only be recovered by subtracting an estimated floor.

This dataset is the teacher, captured on the same lane the candidates are captured on. A root-vs-root self-compare of two cold runs returns exactly 0.0 nats with top-1 agreement 1.000000, so the floor is not estimated and not subtracted — it is zero by construction, and a candidate's raw KLD is its attributable error.

Capture and comparison are separate steps. Comparing two datasets needs neither set of weights, so this capture is reusable by anyone measuring a Qwen3.8-27B derivative without re-paying for the reference forward pass.

The cut

The captured tensor is the model's final hidden state as handed to lm_head — after the text model's final norm, immediately before the head matmul — taken as the head module's input via a register_forward_pre_hook. Replay applies the head only. This is the same cut as k6/tools/hidden_replay.py and as the kimi-k3 hidden-replay convention, which is what makes those artifact families comparable.

Hidden form costs hidden_size * 2 bytes per scored position instead of vocab_size * 4: 10.7 GB here rather than 1.04 TB, a 97x saving. The head is shipped alongside so logits are reproducible from it.

The panel, and why it is this one

panel--qwen38.malaiwah.suite-v5-shard0-1m, the panel 37 existing Qwen3.8-27B registry rows were scored on. Its registry record is marked private with no URI, but the token ids are in fact public and were transported, not reconstructed: they come from malaiwah/qwen38-27b-fidelity-suite-v5 at revision 7797fcce3ffed62b99871348887f4626dc9b2b3b, path suite/tokens/.

Two seals were checked before this capture ran, and both hold:

1. 1.every one of the 512 context files hashes to the token_sha256 recorded for it in the sealed suite manifest; and
2. 2.the sha256 of those 512 digests concatenated in context_index order reproduces the registry panel's sealed panel_token_sha256 caef8a4628d6c07c162100895096f890cdf9cafc8e4c48b3d66035d737ee7cf7.

No tokenizer was run and no text was re-tokenized, so there is no opportunity for a tokenizer-version drift to change the panel.

Reusing the panel does not make these rows rankable against the 37 older ones. Those were measured against a vLLM-captured teacher; this is a transformers capture on the local-cuda-budget lane. The comparability key binds the reference, and the references differ. Holding the tokens fixed means the two groups differ by the lane alone, which makes the difference interpretable — not comparable.

Determinism

Three cold captures were run in three separate processes on one RTX PRO 6000. All three produced the identical capture_content_digest, and the run-1-vs-run-2 self-compare returned exactly 0.0 nats.

Note that transformers reported the fused linear-attention path unavailable (flash-linear-attention / causal-conv1d not installed) and fell back to the reference torch implementation. That fallback is part of this lane's identity: installing those kernels would be a different lane and is not guaranteed to reproduce these digests.

Reproduce

git clone https://github.com/malaiwah/quant-fidelity-suite
cd quant-fidelity-suite
git checkout 9133f4288838a3333b25b575f4d5e4e8ab3b419a

# 1. rebuild the panel tree from the public suite (verifies both seals, no RNG)
python3 k6/tools/build_panel_from_v5_suite.py \
  --tokens-dir <512 fetched suite/tokens/context-*.json> \
  --suite-manifest <suite/shard-0000/suite-manifest.json> \
  --out panel-v5-shard0 \
  --panel-id panel--qwen38.malaiwah.suite-v5-shard0-1m \
  --panel-name "malaiwah Qwen3.8-27B suite v5, shard 0 -- 512 contexts" \
  --expect-panel-token-sha256 caef8a4628d6c07c162100895096f890cdf9cafc8e4c48b3d66035d737ee7cf7 \
  --corpus-revision 7797fcce3ffed62b99871348887f4626dc9b2b3b

# 2. capture
python3 bin/fidelity_dataset.py capture \
  --out qwen38-root --form hidden --role root --lane local-cuda-budget \
  --engine hf-transformers \
  -- \
  --model Qwen/Qwen3.8-27B \
  --model-revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 \
  --weights-repository Qwen/Qwen3.8-27B \
  --model-ref model--qwen.qwen3.8-27b \
  --panel panel-v5-shard0 --panel-role final \
  --panel-id panel--qwen38.malaiwah.suite-v5-shard0-1m \
  --panel-name "malaiwah Qwen3.8-27B suite v5, shard 0 -- 512 contexts" \
  --panel-repository malaiwah/qwen38-27b-fidelity-suite-v5 \
  --panel-revision 7797fcce3ffed62b99871348887f4626dc9b2b3b \
  --tokenizer-id Qwen/Qwen3.8-27B \
  --dataset-id fidelity--qwen38-27b.malaiwah.root.bf16 \
  --dataset-name "Qwen3.8-27B BF16 root fidelity dataset (hidden form), suite-v5 shard-0 panel" \
  --repository malaiwah/qwen38-27b-fidelity-root-v1 \
  --author malaiwah --run-name hf-capture --cold-run cold-run-1 \
  --device cuda --device-map '{"":0}' --dtype bfloat16

# 3. check the seal
python3 bin/fidelity_dataset.py verify qwen38-root

Environment: transformers 5.8.1, torch 2.11.0+cu130, one NVIDIA RTX PRO 6000 Blackwell (96 GB), driver 595.58.03, bfloat16, whole model resident on one device. 512 windows in 335 s, about 0.65 s/window.

Using it

# a candidate captured the same way, then:
python3 bin/fidelity_dataset.py compare \
  --reference <this dataset> --candidate <candidate dataset> \
  --out report --vocab-chunk 24832

The comparison reports mean tokenwise KLD in nats, top-1 agreement, and a registry receipt. Because both sides are captured by the same engine on the same lane, the floor is structurally zero rather than subtracted.

Full specification: <https://github.com/malaiwah/quant-fidelity-suite/blob/main/docs/FIDELITY-DATASET-SPEC.md>