tetracta/llm-xray-lesion-scans

<!-- vg1-card-status-20260922 -->

VG1 status · 22 September 2026 (Europe/Istanbul)

The 7B class opened on 21 September 2026. Any registered account may scan the repositories on the eligible list of the scope page — exact Apache-2.0 revisions, listed there with their status — within the free allowance of 20 browser scans and five distinct source models per calendar month. 7B-class repositories run as single-model quantization simulations. A comparison of two 7B-class checkpoints is currently accepted by the submission form but fails inside the worker sandbox (under investigation, 22 September 2026); please do not submit one until this sentence is removed (a job that fails this way is recorded as failed and the scan it reserved is returned automatically). HuggingFaceTB/SmolLM3-3B-Base and HuggingFaceTB/SmolLM3-3B are listed on the register, but the current instrument release does not support their tokenizer contract, so those submissions fail as well. Scans run on Tetracta-operated local GPU workers; no cloud GPU is used in this beta. Models above the 7B class are not scanned in this beta. This card's presence is not an eligibility grant. External customer acceptance remains pending; results are research-stage and payments are disabled.

A report describes its recorded artifacts and measurement conditions; it is not a model-quality ranking, safety certificate or deployment verdict. A quantization simulation does not create a deployable quantized model. Knowledge scans remain unavailable and earlier withdrawn claims remain withdrawn. Hallucination, undertraining and overtraining remain research questions, not measured product features.

Service status · Current scope and report guide · Correction record · Report examples <!-- /vg1-card-status-20260922 -->

<!-- vg1-card-status-20260912 -->

Model X-Ray status · 12 September 2026 (Europe/Istanbul) (12 September 2026 — superseded by the 22 September entry above; "Customer jobs run on RunPod" no longer applies.)

VG1 has been deployed. Customer scan-to-report acceptance is still pending.

The free first-beta scope covers eligible public models up to and including 7B, within each account's allowance, for checkpoint comparisons and in-memory quantization simulations. Customer jobs run on RunPod. The service selector determines supported models, revisions and scan types; this card's presence is not an eligibility grant. Later Pro access above 7B requires an explicit grant and the enabled account limits.

Knowledge scans remain unavailable. Reports describe recorded artifacts and measurement conditions, not model-quality rankings, safety certificates or deployment verdicts. A quantization simulation does not produce a deployable quantized model. Hallucination, undertraining and overtraining remain research questions, not measured product features. Earlier withdrawn X-Ray interpretations remain withdrawn; the historical study and model measurements below are not new VG1 customer results.

Service status · Current scope and report guide · Dated correction <!-- /vg1-card-status-20260912 -->

Legacy Model X-Ray materials — withdrawn

Correction dated 6 September 2026.

The Model X-Ray results previously distributed from this repository have been withdrawn. These files are historical artifacts, not current evidence, and must not be used to support location, knowledge, portrait, lesion-response, simulated-quantization, quality, safety or deployment claims.

No replacement figures are published. Validation remains pending.

Correction record: https://www.tetracta.ai/model-xray/correction/

— Tetracta

📚 Current scope and report guide: https://www.tetracta.ai/model-xray/scope/

Current availability: see the dated VG1 status above. Customer scan-to-report acceptance remains pending.

Historical documentation

The protocol, numbers and example report below belong to the withdrawn study. Their legacy risk grades, robustness interpretations and production-version wording are not current VG1 results or capabilities. The unchanged data files are retained for traceability. Current report examples: https://huggingface.co/spaces/tetracta/model-xray-sample-reports

<details> <summary>Original study documentation — withdrawn, historical</summary>

LLM Lesion X-Ray — six 1B-class models, one protocol

Layer-lesion scans of six 1B-class language models, produced with a single pre-registered protocol. This repository contains measurements, not weights.

Two of the six models are a matched pair: a constant-state architecture and a transformer, trained by us on identical data with the same budget. On a standard benchmark they land 0.62 macro points apart (95% CI [−0.06, +1.28] — statistically equal). Under this instrument they are not remotely the same model.

Why this exists

A benchmark asks what does the model know. It cannot ask how is it built — not because nobody tried, but by construction: it only ever sees outputs.

This dataset is what the second question looks like when you answer it with a crude, hard-to-game instrument.

The instrument

For one layer at a time:

1. 1.Record the intact model's next-token distribution over the full output space, on 12 fixed English prompts → p
2. 2.Damage that layer — zero the smallest 5 / 10 / 20 / 30 % of its weights by magnitude, or add Gaussian noise at σ = 0.005 / 0.01 / 0.02 × weight standard deviation
3. 3.Record the distribution again → q; score = KL(p‖q)
4. 4.Restore the weights exactly, move to the next layer

Nothing is written to disk during a scan; checkpoint hashes were verified unchanged after every run.

Two quantities fall out:

The arms

van-1b and znext-v43 were trained by us on identical data. They are the matched pair; the only variable between them is architecture.

Headline numbers

At 30 % pruning, mean KL per layer (×10⁻³, lower is sturdier):

Three things a reader may find useful — including one that argues against the authors:

1. 1.The matched pair differs by 7.9× (base) / 9.1× (after SFT) in fragility and 37 % vs 72 % in concentration, while sitting 0.62 macro points apart (95% CI [−0.06, +1.28]) on a standard benchmark.
2. 2.Concentration barely moves under fine-tuning (37→39, 72→69). Whatever sets it appears to be set by the end of pretraining.
3. 3.Fine-tuning made Qwen more robust (18.5 → 10.3) and made both of our models more fragile. We publish this because it is what we measured.

[image]

[image]

What this dataset does not establish

Attributing an internal difference between two checkpoints to architecture requires independent-seed controls — several models from the same recipe with different seeds, to bound run-to-run variation. Those controls were not run for this release.

We know this failure mode first-hand. In July 2026 we published internal differences between two arms and retracted them after three purpose-trained control models showed the figures sat inside ordinary run-to-run noise.

The gaps here are 8–20×, not noise-band. That makes them more interesting. It does not make them controlled. Read every number as

"these checkpoints differ by X under this instrument"

and not as an architectural claim.

For the Qwen arms the limitation is stronger: data, scale and architecture all differ at once, and no reading of them can be decomposed.

File format

{
  "etiket":  "Z-Next v4.3-1B (TABAN)",
  "n_katman": 24,
  "n_istem":  12,
  "tohum":    1234,
  "budama":  {"0.05": [...], "0.1": [...], "0.2": [...], "0.3": [...]},
  "gurultu": {"0.005": [...], "0.01": [...], "0.02": [...]}
}

Field names are Turkish: etiket = label, n_katman = layer count, n_istem = prompt count, tohum = seed, budama = pruning, gurultu = noise. Each array holds one KL value per layer, ordered input → output.

To compare across architectures, normalise the layer index to 0–1 — the models have different depths and raw indices are not comparable.

Reproduce the headline numbers

import json, glob

for path in sorted(glob.glob("scans/*.json")):
    d = json.load(open(path))
    p = d["budama"]["0.3"]                       # 30% pruning profile
    n = len(p)
    fragility     = sum(p) / n
    concentration = sum(sorted(p, reverse=True)[: n // 5]) / sum(p)
    print(f"{d['etiket']:32s} fragility {fragility*1e3:6.1f}e-3   "
          f"concentration {concentration:5.1%}")

Plot a depth profile

import json, matplotlib.pyplot as plt

for name in ("znext-v43-base", "van-1b-base"):
    d = json.load(open(f"scans/{name}.json"))
    p = d["budama"]["0.3"]
    x = [i / (len(p) - 1) for i in range(len(p))]   # normalised depth 0–1
    plt.plot(x, p, marker=".", label=name)

plt.xlabel("normalised depth"); plt.ylabel("KL after damaging this layer")
plt.legend(); plt.show()

Example report

example/example_report_znext_base_to_sft.html is a real report from our X-ray service, produced by the unmodified production pipeline, comparing our base model against its fine-tuned version.

It answers a different question from the scans above: not "how robust is this model" but "what did fine-tuning change inside it".

The reading: fine-tuning reshapes the interface (upper layers), not the foundation.

Open the retained historical file in a browser. Current service status (customer acceptance pending): → <https://www.tetracta.ai/xray.html>

Protocol

PROTOCOL.md — including the reading rules and the declared failure condition ("if the arms are indistinguishable, that is a result and will be reported as one") — was written before the runs, not after seeing them.

Related

• —Full research note — <https://www.tetracta.ai/note-xray-lesion.html>
• —Current service status and supported scan types — <https://www.tetracta.ai/xray.html>
• —Live demo of the constant-state model, one consumer GPU — <https://www.tetracta.ai/zchat>
• —Earlier matched-pair study with seed-null controls — <https://huggingface.co/tetracta/llm-xray-twin-study-1b>

Citation

@misc{tetracta_lesion_xray_2026,
  title  = {LLM Lesion X-Ray: six 1B-class models under one pre-registered protocol},
  author = {Tetracta AI},
  year   = {2026},
  url    = {https://huggingface.co/datasets/tetracta/llm-xray-lesion-scans}
}

</details>