thaki-AI/daily-paper-2026-07-08-attested-confidential-sovereign-inference

Attested Confidential Sovereign Inference

TL;DR — Attested Confidential Inference (ACI) cryptographically binds GPU TEE remote-attestation evidence to signed model-provenance records, enforcing the binding at Kueue job-admission so every inference request carries a regulator-verifiable proof linking enclave identity, tenant identity, and audited weight digest.

ThakiCloud AI Research · 2026-07-08 · 📝 Tech blog (KO)

Problem

Sovereign on-premises AI platforms serving regulated sectors cannot cryptographically prove to a regulator that (a) tenant data never left a GPU confidential-computing enclave, and (b) the exact audited model weights served each request. GPU TEEs provide attestation, and model-provenance work provides signed release records, but the two are never cryptographically joined.

Approach

ACI binds RATS-style remote attestation evidence to a signed model-provenance ledger entry and enforces the binding at the Kubernetes job-admission boundary via Kueue. Five logical components (Provenance Registrar, Attestation Broker, Binding Ledger, Kueue Admission Gate, Regulator Verifier API) produce a per-request signed binding entry B that links H(A) || H(P) || tenant || req_id.

Key contributions

• —Platform contribution: turns Keycloak/Kueue multi-tenant isolation primitives into a regulator-auditable cryptographic guarantee by adding attestation-to-provenance binding on a real Keycloak/Kueue/ArgoCD/PostgreSQL/NATS stack.
• —Societal contribution: lowers the trust barrier for health, government, and finance sectors to adopt on-premises LLM inference without trusting the platform operator — the proof does not depend on operator honesty.
• —Scientific contribution: first work to compose GPU-TEE remote attestation with signed model-provenance verification as a per-request, regulator-verifiable gate enforced at scheduler-admission time inside a multi-tenant Kubernetes+Kueue system.

Figures

[image] Per-request software binding overhead; total ~0.01 ms = 0.0002% of the 4286 ms H200 inference baseline. _{Measured on 4x H200-NVL / AMD EPYC 9335 (CC-mode OFF)}

[image] Attestation handshake cost per request falls below 1e-4 ms once amortized across a batch. _{Measured software attestation flow (excludes GPU-TEE HW evidence)}

[image] The only material overhead is the GPU CC-mode throughput penalty (cited); ACI's software binding path is negligible (measured). _{Software measured on H200; GPU-TEE penalty is cited literature (4-8%), not measured}

Results (as argued)

Software binding path measured on a real NVIDIA H200 cluster (4x H200-NVL, AMD EPYC 9335, tkai-prod-compute-h200, CC-mode OFF). Baseline L_base = 4286 ms/req (vLLM, Qwen3.6-35B-A3B, 128 tok, 29.9 tok/s). Signed ledger append 0.0037 ms/req (signing overhead 0.0014 ms); per-request provenance-digest verify 0.00014 ms (one-time full 512MB shard SHA-256 at 2.15 GB/s, amortized); software attestation flow 0.0062 ms/req, amortizing to 0.00005 ms/req at N=128. Combined ACI software binding adds ~0.01 ms/req = 0.0002% of baseline latency (<0.001%), negligible. The intrinsic GPU-TEE throughput penalty is NOT measured (CC-mode was disabled) and remains cited-only: 4-8%, shrinking with batch/input size (chrapek2025confidential, zhu2024hopperbenchmark); CPU-TEE latency overhead ~20%.

Limitations

Only the GPU-TEE hardware layer remains unmeasured: CC-mode was disabled cluster-wide, so the intrinsic TEE throughput penalty (cited 4-8%) and hardware attestation-evidence generation could not be exercised and stay source-cited - the sole outstanding future measurement. The software protocol itself is now measured on real H200 hardware. Attestation-to-load TOCTOU gap and supply-chain custody of the release signing key are acknowledged limitations. Ledger reconciliation provides detection rather than operator-proof prevention for out-of-band shadow requests.

Abstract

Sovereign, on-premises AI platforms serving regulated sectors must answer a question their operators cannot answer today: can they prove to a regulator that a tenant's data never left a confidential-computing (CC) enclave, and that the exact audited model weights served each request? GPU trusted execution environments (TEEs) now provide the first half through remote attestation, and model-provenance work provides the second through signed release records, but the two are treated as separate concerns and are never cryptographically joined. This paper is primarily an analytical, design-oriented contribution, but we now also report real measurements of its software-side binding path. On a real NVIDIA H200 cluster (four H200-NVL GPUs, AMD EPYC 9335) we measured the ACI software binding overhead - signed ledger append, per-request provenance-digest verification, and the software attestation flow - and find it adds roughly 0.01 ms per request, about 0.0002% (well under 0.001%) of the measured 4286 ms baseline inference latency: negligible. What remains unmeasured is the GPU-TEE hardware component: our cluster ran with confidential-computing (CC) mode disabled, so the intrinsic TEE throughput penalty and hardware attestation-evidence generation are still reported only as ranges explicitly attributed to prior benchmark studies, never as our own measurements. We present Attested Confidential Inference (ACI), a protocol that binds RATS-style remote attestation evidence to a signed model-provenance ledger entry and enforces the binding at the Kubernetes job-admission boundary, so that

Files

• —LaTeX source
• —References (BibTeX)

Citation

@techreport{thaki_attested_confidential_sovereign_inference_2026,
  title  = {Attested Confidential Sovereign Inference},
  author = {ThakiCloud AI Research (Hyojung Han)},
  year   = {2026},
  institution = {ThakiCloud}, note = {thaki-AI/daily-paper-2026-07-08-attested-confidential-sovereign-inference}
}

Generated by ThakiCloud nightly research pipeline. License: CC BY 4.0.