SZLHOLDINGS/alloy-sovereign-eval-runs

<div align="center"> <p>

  

</p> </div>

Alloy Sovereign Eval Runs · the honest first measured run

Append-only measured eval runs produced by routing SZL's K-Verify Benchmark v1 through the live Alloy governed-inference stack on SZL's own sovereign metal (provider: sovereign, zero cloud, zero spend). Each row is one inference: its verdict, latency, NVML-measured energy, and a signed receipt id that is re-checkable against the live Alloy receipt chain.

Built and maintained by SZL Holdings. Apache-2.0.

Honesty first — read this before citing any number. This first run scored 13/25 = 52% accuracy. That number is published as measured, un-cherry-picked: it is the deterministic first 25 items of k_verify_v1.jsonl (kv-001…kv-025), a single pass, no retries, no re-runs, and no model-graded scoring — only deterministic string/numeric comparison. Nothing here is upgraded into a stronger claim.

### ⚠ This is NOT a benchmark of SZL-1 This run was served by the laptop node (gpu2) on the small base model `qwen2.5:3b` — because during the run the tower node was DOWN (HTTP 530, tunnel had no live connector). qwen2.5:3b is not SZL-1 (the owner's QLoRA fine-tune llama3-szl-finetuned-q4, which lives only on the tower) and it is not szl-nemo. These 25 rows must never be cited as a benchmark of SZL-1 or szl-nemo. SZL-1 has no measured benchmark yet — that stays honestly UNKNOWN until it is served and measured.

The measured result (MEASURED)

Slice: deterministic first 25 items of k_verify_v1.jsonl (kv-001…kv-025). Served by: sovereign#laptop · model qwen2.5:3b · demo=false · single pass. Source of truth for expected answers: the public `SZLHOLDINGS/k-verify-benchmark-v1` (raw model text is not published here — only its response_hash sha256).

Verdict table (MEASURED)

By answer type (MEASURED)

By category: all 25 items in this deterministic first-25 slice fall in the benchmark's STEM block (stated plainly; this is a slice artifact, not a chosen subset).

Latency & energy (MEASURED)

Every one of the 25 rows carries energy.status: "MEASURED" from the NVML board-level counter on meter2 — no joule figure is fabricated. Energy is a whole-GPU window (not per-process attribution), labelled as such.

Receipts (MEASURED)

25 / 25 receipts SIGNED. Every row's receipt_id is a signed governed-inference receipt on the live Alloy receipt chain — re-checkable, not just asserted.

Schema — runs/<date>-<node>-<model>.jsonl

One JSON object per line. Published fields are exactly (private working fields _output / _expected are stripped — raw model text is never published):

{
  "benchmark_id": "kv-001",
  "category": "STEM",
  "answer_type": "numeric",
  "model": "qwen2.5:3b",
  "servedNode": "laptop",
  "servedProvider": "sovereign",
  "response_hash": "sha256:...",
  "verdict": "CORRECT",
  "latency_ms": 1366,
  "energy": { "status": "MEASURED", "joules": 20.209, "avgWatts": 21.27,
              "windowMs": 950, "meterHost": "meter2.a-11-oy.com" },
  "cost_status": "UNKNOWN",
  "receipt_id": "…",
  "receipt_status": "SIGNED",
  "demo": false
}

Grading rules (verbatim from the locked eval design)

Grading is deterministic MEASURED string/numeric comparison, NO model-graded scoring:

• —numeric: strip commas/spaces from output; if the expected numeric value appears among output numbers → CORRECT; a different number present → INCORRECT; no number at all → UNPARSEABLE.
• —exact: case-insensitive; expected string present as substring in output → CORRECT; non-empty non-match → INCORRECT; empty output → UNPARSEABLE.
• —UNPARSEABLE stays UNPARSEABLE (never coerced to correct/incorrect).

Prompt wrapper (identical for every item):

Answer the following question with ONLY the final answer, as briefly as possible.
If you do not know, answer exactly UNKNOWN.

Question: <q>

Transport: POST /api/run, body {goal, provider:"sovereign"}. Sovereign ONLY, zero cloud, zero spend. Single pass — no verify, no retries, no re-runs. Paced ≥ 4 s/item (well under the rate limit).

Load it

from datasets import load_dataset

ds = load_dataset("SZLHOLDINGS/alloy-sovereign-eval-runs", "runs", split="train")
print(ds[0])   # one measured eval row

Or pull the raw JSONL directly (always works):

from huggingface_hub import hf_hub_download
import json
p = hf_hub_download("SZLHOLDINGS/alloy-sovereign-eval-runs",
                    "runs/2026-07-12-laptop-qwen2.5-3b.jsonl", repo_type="dataset")
rows = [json.loads(l) for l in open(p)]
print(len(rows), "rows")          # 25

Verify it yourself — the receipts are real

Every receipt_id is a signed governed-inference receipt. Re-check them against the live Alloy receipt chain — don't trust this card:

• —Trust center / receipt surfaces: <https://a-11-oy.com>
• —Live Alloy app (custom domain): <https://a11oy.net>
• —Cross-check the expected answers in the open source benchmark: `SZLHOLDINGS/k-verify-benchmark-v1` (k_verify_v1.jsonl sha256 dc0a12efd873f6b9174e9c247c9fe5158b21a681514dc79637edafb198412915).
• —Re-check any receipt offline against the open governed-receipt-spec with the dependency-free governed-receipt-verifier.

The response_hash lets you confirm you have the exact byte-string the receipt was signed over, without the raw text ever leaving SZL's metal.

Governance

szl-governance:
  doctrine: v11 LOCKED
  lambda:
    label: "Λ = Conjecture 1 — advisory governance floor, never proven"
    proven: false
    trust_ceiling: 0.97          # never 1.00 / 100%
  locked_proven_formulas:
    count: 8                     # machine-enforced theorem locked_count_eight (no axiom)
    set: [F1, F4, F7, F11, F12, F18, F19, F22]
    source: "szl-holdings/lutar-lean PROVEN_FORMULAS.md"
  honesty:
    every_value_labelled: [MEASURED, REPORTED, DECLARED, UNKNOWN, UNAVAILABLE]
    fabrication: forbidden
  run:
    served_by: "sovereign#laptop (gpu2)"
    model: "qwen2.5:3b"          # NOT SZL-1, NOT szl-nemo
    is_benchmark_of_SZL1: false
    tower_status_during_run: "DOWN (HTTP 530, tunnel had no live connector)"
    accuracy: "13/25 = 52% (measured, single pass, no retries)"
    energy: "MEASURED (NVML meter2, whole-GPU window), total 286.023 J"
    receipts: "25/25 SIGNED"
    demo: false
    cost: UNKNOWN                # own-metal


Citation

Cite this. Part of the SZL Holdings Ouroboros Thesis (Governed Post-Determinism). Concept DOI (always-latest): 10.5281/zenodo.19944926. Author: Stephen P. Lutar Jr. · ORCID 0009-0001-0110-4173 · License Apache-2.0. Full DOI-pinned lineage (v1→v26) + the 8 papers: szl-papers PAPERS_INDEX.

Honesty (Doctrine v11): Λ unconditional uniqueness is Conjecture 1 (machine-checked FALSE as stated) — never a theorem; conditional uniqueness is Theorem U (axiom-free). Locked-proven formulas = exactly 8 {F1,F4,F7,F11,F12,F18,F19,F22}; ~185 experimental theorems are a separate CI-green tier; Khipu BFT safety = Conjecture 2. Trust never 100%.

@misc{lutar_szl_ouroboros,
  author    = {Lutar, Stephen P., Jr.},
  title     = {SZL Holdings --- The Ouroboros Thesis (Governed Post-Determinism)},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.19944926},
  url       = {https://doi.org/10.5281/zenodo.19944926},
  note      = {Concept DOI --- always resolves to the latest version. ORCID 0009-0001-0110-4173. CC-BY-4.0.}
}

Signed-off-by: Stephen Lutar <stephenlutar2@gmail.com>

◇ Explore the SZL Holdings estate

▶ a11oy console (a-11-oy.com) · live app (a11oy.net) · a11oy Space · K-Verify Benchmark v1 · energy-attested-runs · all datasets & models → SZLHOLDINGS · GitHub org

<div align="center">

[🛡️ SZLHOLDINGS on Hugging Face →](https://huggingface.co/SZLHOLDINGS) · [a-11-oy.com →](https://a-11-oy.com) · [Estate hub — live →](https://szlholdings-szl-estate-live.static.hf.space)

Governed AI you can prove.

_{SLSA: L1 honest · L2 attested · L3 roadmap. Λ = Conjecture 1. Trust ceiling 0.97. Labels honest by default.}

</div>