malaiwah/qfs-smollm2-135m-wikitext2-campaign-v1

SmolLM2-135M QFS calibration and evaluation campaign

A small stored-weight fidelity study, not a broad model-quality benchmark.

• —Evaluation: 16 complete WikiText2 raw test articles, one 256-token window each, 4080 prediction positions.
• —Calibration: 32 disjoint train articles, 256 tokens each, 8192 calibration tokens.
• —Complete article title, normalized content and exact 13-token-ngram separation were checked. Validation is unused. Pretraining overlap remains unknown.
• —Original model/tokenizer: HuggingFaceTB/SmolLM2-135M@93efa2f097d58c2a74874c7e644dbc9b0cee75a2.
• —Source corpus: Salesforce/wikitext@b08601e04326c79dfdd32d625aee71d232d685c3, wikitext-2-raw-v1; original Wikipedia contributors and WikiText/Salesforce attribution retained. Corpus derivatives are not exclusively MIT.

Reuse

The raw QFS panel lives under panel/ (panel.json, its sealed build receipt, referenced arrays and source mapping). Use that path, not a capture's internal panel view. prepared/selection.receipt.json binds exact calibration/evaluation row order, token IDs, masks, article hashes, extraction intervals and the separation report. provenance/ retains genuine pinned Hub metadata and download identities; source/ retains the deterministic driver and actual optimizer instrumentation.

Source terms: https://creativecommons.org/licenses/by-sa/3.0/ and https://www.gnu.org/licenses/fdl-1.3.html. Model weights use the upstream Apache-2.0 declaration; that does not relicense the corpus.

No source URL pretends that derived local article files existed upstream: panel-provenance/campaign-source-mapping.json maps each to its real Parquet path, immutable hash and row interval.

source/campaign.py is the canonical conversion/panel driver; operational sidecars stay outside the sealed panel tree. source/RUN.txt and source/requirements-calibration.freeze.txt record the exercised environment and its limits. Original source versions remain available through this repository's immutable history. The metadata-only scope corrections preserve the original quantizer receipts and all weight bytes.

Measured results — 2026-09-08

All variants use the same 16 evaluation articles and 4,080 prediction positions. Each workflow captured its weights twice in fresh processes; every within-variant reproduction comparison, including the native BF16 control, measured exactly zero KL.

GPTQ g64 reduced observed mean KL by 30.90% versus matched RTN g64; g32 reduced it by 41.54%. Both were lower than RTN on all 16 selected articles. This is a small, author-reported stored-weight reconstruction study: weights were decoded to BF16 for eager CPU forward, not evaluated with native quantized GEMM. The registry retains advisory status, unknown cross-stack bias, and independently_verified=false. Reproduction controls are not transferable bias corrections.

• —Full analysis, paired exploratory bootstrap intervals, timing breakdowns, provenance and costs
• —SVG, CSV, plot data, and immutable interactive/export links.
• —Registry snapshot containing all three accepted measurements. Original requests: RTN g64, GPTQ g64, GPTQ g32.
• —Canonical captures, both original captures/reproduction evidence, source pins and exact comparison-file hashes are linked separately for every variant in the analysis.

Nine trained-campaign Jobs, including five failed diagnostic attempts, reserved a combined maximum compute estimate of $0.099. Their observed running times imply $0.0115 using the original $0.03/hour quote and upward minute rounding. This is not an HF invoice; storage, transfer, credits and local calibration electricity are excluded. The broader hardening trial's reservation ledger remains below its $5 compute ceiling.

Bootstrap intervals are nominal, exploratory article-cluster resampling over only 16 selected articles, not a calibrated 95%-coverage or generalization claim. Original comparison receipts retain their recorded uncertainty fields unchanged. No serving-speed, GPU portability, model-utility or production-ranking claim is made.

Recompute the mean KL and paired bootstrap summaries without an account, model download or compute rental:

python source/replay_analysis.py reports/trained-study-20260908/analysis.json

This reads the original public comparison receipts at their exact commits and checks their file hashes. It requires NumPy and huggingface_hub; it does not rerun inference or verify an invoice.