Corbenic/galahad-bench

A 12B model beats frontier models at 0 tokens — on work it has already verified

100% accuracy · 0 generation tokens · bit-exact · forever.

A frozen 12B model with a memory of verified solutions answered 180/180 fresh instances across nine problem families — at zero generation tokens, bit-exact, in 6–23 ms, at 36 mWh per answer. Same outcome on four models from four vendors, dense and mixture-of-experts.

Scope, plainly: on cold-start reasoning, frontier models are clearly ahead of any 12B — the report says so. The inversion is on solved-and-verified work, and that territory only grows.

On everything already solved, a frontier API pays a full generation pass on every query, forever. Verified reuse costs zero tokens, returns in milliseconds, and produces identical bits every time. Break-even on the one-time deposit: 17–34 queries.

Vector search fails here — measured. On a 4,500-item verified store, approximate similarity retrieval selects the wrong item 94.3% of the time. Exact addressing: zero collisions. It doesn't degrade gracefully; it silently returns a wrong, verified-looking answer.

Not a cache. What's stored is a verified method, so it answers inputs it has never seen, and separate items combine into answers never deposited. Emptied of memory, the system solves nothing — the capability is the store, not the weights.

The public testbench opens with the paper. Leave your email on the page and you're first in line.

A Frozen 12B Beats Frontier Models on Verified Work. Industry experience report · Sietse Schelpe — Corbenic AI. System: Galahad. Storage/integrity engine: Merlin.

_{zero-token LLM inference · deterministic LLM output · bit-exact reproducible inference · verified LLM memory · frozen model · LLM inference cost reduction · RAG alternative · semantic cache · vector retrieval failure · exact content addressing · KV cache reuse · auditable AI · reproducible AI for regulated industries · LLM energy per query · Gemma 4 12B · Claude Fable 5 comparison}