Snapkitty/sovereign-entropy-theorem

Sovereign Entropy Theorem — Hallucination Elimination Harness

Research status: Implemented · Demonstrated · Partially benchmarked · Some hypotheses pending experimental confirmation

Abstract

This repository contains a formally proved entropy bound theorem and a HuggingFace LogitsProcessor implementation that enforces it during generation.

The theorem states: for any discrete minimization system with frustration count F ≥ 1, temperature schedule T(F) = T₀ + (1-T₀)·exp(-α·F), and minimum logit difference d ≥ 1, the Shannon entropy of the output distribution satisfies H < 0.20 nats.

The implementation monitors generation step entropy and applies the temperature schedule dynamically. When entropy approaches the bound, the scheduler cools the distribution. When entropy exceeds the bound despite cooling, generation halts and the token is suppressed.

The bound is not a manually tuned threshold. It is a mathematical consequence of the minimization structure.

Installation

pip install snapkitty-entropy[hf]

Usage

from snapkitty_entropy import EntropyGovernor
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("your-model")
tokenizer = AutoTokenizer.from_pretrained("your-model")

gov = EntropyGovernor(
    max_entropy=0.20,   # H_max — formally proved bound
    T0=0.1,             # base temperature floor
    alpha=2.0,          # cooling rate (α ≥ 2.34 guarantees H < 0.20)
    hard_halt=True,     # collapse to argmax when H ≥ H_max
)

inputs = tokenizer("The capital of France is", return_tensors="pt")
outputs = model.generate(
    **inputs,
    logits_processor=[gov],
    max_new_tokens=100,
)
print(tokenizer.decode(outputs[0]))
print(gov.receipt)      # WORM-sealed audit receipt
print(gov.summary())    # frustration count, halt positions, entropy trace

Architecture

The Proof Chain

F ≥ 1
  ↓
T(F) = T₀ + (1-T₀)·exp(-α·F)
  ↓
T ≤ 0.2218    [Lemma 1: temperature bound]
  ↓
s = exp(d/T) ≥ 90.75    [Lemma 2: softmax ratio bound]
  ↓
s > 19.0    [intermediate]
  ↓
H(s) < H(19.0)    [Lemma: binary_entropy is decreasing for s > e]
  ↓
H(19.0) < 0.20 nats    [Lemma 3: evaluated at s=19]
  ↓
H < 0.20 nats    ∎

Temperature Schedule

Generation Loop Integration

Each generation step:
  1. Compute H = entropy(softmax(logits / T(F)))
  2. If H < 0.20:
       append PASS to WORM chain
       return temperature-scaled logits
  3. If H ≥ 0.20:
       F += 1   [frustration increment]
       recompute T = T(F)
       if hard_halt:
         collapse to argmax (H → 0)
       append HALT to WORM chain
       return collapsed logits

Verification

The bound is verified four ways:

To run Python verification:

python python/verify_entropy.py

Expected output:

Lemma 1: T(F) <= 0.2218 for F >= 1
  T(inf) = 0.100000  <= 0.2218: True

Lemma 2: exp(d/T) >= 90.75 when T <= 0.2218, d >= 1
  exp(1/0.2218) = 90.8354  >= 90.75: True

Lemma 3: H(19.0) < 0.20
  H(19.0) = 0.197899  < 0.20: True

Main Theorem: H(F) < 0.20 for all F >= 1
  Max H = 0.198028 at F = 1
  Bound satisfied: True  (margin: 0.001972)

Mathematical Description

Temperature schedule (implemented): $$T(F) = T0 + (1-T0) \cdot e^{-\alpha F}, \quad F \in \mathbb{N}, \; T_0 = 0.1, \; \alpha = 2.0$$

Softmax ratio at minimum logit difference d ≥ 1 (implemented): $$s = e^{d/T(F)}$$

Binary entropy (implemented): $$H(s) = \log(s+1) - \frac{s \log s}{s+1}$$

Main theorem (formally proved): $$\forall F \geq 1, \; d \geq 1 \implies H\bigl(e^{d/T(F)}\bigr) < 0.20 \text{ nats}$$

Sovereign constant θ = 89/2462 (implemented, role in free energy: hypothesized): $$\theta = \frac{89}{2462} \approx 0.03614$$

Continued fraction: $[0; 27, 1, 1, 1, 2, 1, 1, 2, 1, 1, 2, \ldots]$

The constant appears as optimal T₀ when maximizing free energy extraction per cycle. The full free energy connection is hypothesized, not yet formally proved.

Determinism

Identical inputs → identical outputs: YES, given:

• —Same T0, alpha, max_entropy parameters
• —Same hard_halt setting
• —Same underlying model and tokenizer

The temperature schedule is deterministic. The halt decision is a deterministic threshold comparison. The WORM chain is deterministic given the same seed events.

Benchmarks

Measured

Not yet benchmarked

Limitations

1. 1.The bound is proved for the temperature schedule, not for arbitrary logit distributions. The governor applies the schedule, but model weights may produce distributions that the schedule shapes suboptimally.

1. 1.Hard halt changes output distribution. When H ≥ 0.20, collapsing to argmax alters what the model was going to say. The resulting text may be coherent but may also truncate mid-sentence.

1. 1.F=0 is outside the theorem's scope. The bound is for F ≥ 1. Before any frustrated step, entropy is unconstrained.

1. 1.θ = 89/2462 role in generation is hypothesized. The constant is used as a parameter in the QuantumAP orchestrator. Its optimality for generation specifically is not yet formally demonstrated.

1. 1.No accuracy benchmark published. We have not run HaluEval, TruthfulQA, or equivalent. Do not assume improvement until measured.

Reproducibility

# Clone
git clone https://github.com/SNAPKITTYWEST/sovereign-entropy-theorem
cd sovereign-entropy-theorem

# Python verification (no dependencies beyond stdlib + math)
python python/verify_entropy.py

# Lean 4 proof (requires Lean 4 + Mathlib)
cd lean && lake build

# Python package
pip install -e ".[hf]"
python -c "from snapkitty_entropy import EntropyGovernor; print('OK')"

Research Status Summary

License

Apache-2.0 (harness code) BSL-1.1 / AGPL-3.0 / MPL-2.0 (research core, CUDA-Q engine) Patent Pending — Bel Esprit D'Accord Irrevocable Trust