spinozans/emender-e97-4b-gguf

EMENDER E97 4B — GGUF (llama.cpp)

GGUF conversions of the EMENDER E97 4B model family for llama.cpp. E97 is a 4,045,972,080-parameter hybrid recurrent architecture (bounded tanh-state sequential recurrence, 18 layers, dmodel 3840, 60 heads x 64, SwiGLU MLP hidden 9600, tied embedding/lmhead, vocab 50,281 p50k_base + 24 codex space-run extras). It has NO attention and NO KV cache — state is bounded per layer, which is why it runs efficiently on CPU.

Two checkpoints are provided:

1. 1.`e97-4b-base-99B-f32` / `e97-4b-base-99B-q4_0` — the raw from-scratch pretraining checkpoint (~99.7B tokens). Pure LM continuation model; no post-training, no chat training.
2. 2.`e97-4b-pi-f32-aligned` / `e97-4b-pi-q8_0-v2` / `e97-4b-pi-q4_0` — the verified Pi-native tool-agent checkpoint (the dual-gate-passed, promoted v6-u96), post-trained for a coding-agent tool surface.

Quantification — read before choosing

Every quant was qualified on a frozen 32-prompt panel against its own f32 reference (top-1 next-token agreement, KL over top-64 logits, greedy-exact):

\* 32 CPU threads, shared host. f32 runs at ~7-8 tok/s.

Honest guidance: q80-v2 is the smallest quant that passes the port's qualification precedent (top-1 >= 0.97, KL <= 0.05). The q40 files are provided as small-footprint options with their measured degradation stated — the agent q40 misses by exactly one prompt; the base q40 degrades more.

Running (requires the emender_e97 llama.cpp arch)

These GGUFs use a registered custom architecture (emender_e97); use a llama.cpp build with that arch registered (the Emender repo carries the registration patch series). Standard invocation:

llama-cli -m e97-4b-base-99B-q4_0.gguf -p "Once upon a time" -n 64 -t 32 --temp 0.7 -st
llama-server -m e97-4b-pi-q8_0-v2.gguf --host 127.0.0.1 --port 8151 -t 32

Caveats, stated plainly

• —No chat template is embedded. llama.cpp falls back to a ChatML-style wrapper; that is NOT the trained format. The agent checkpoint was trained on a strict five-line Analysis/Commentary/Action/Arguments frame protocol under a canonical system prompt — for on-distribution behavior, use the Emender repo's serving path (OpenAI-compatible server) or the CPU REPL, which implement the trained codec.
• —Greedy decoding repetition-collapses this model (both checkpoints; a model property, reproduced identically across f32 and q4). Sample with temperature ~0.7 / min-p, or use DRY/repetition penalties.
• —The base model is a raw LM: it continues text, it does not chat.
• —The agent checkpoint is a narrow coding/tool agent, not a general assistant.

Provenance

• —Base: from-scratch frontier pretraining run, step 24448 (~99.72B tokens), checkpoint SHA-256 3ace004251643acf2e7c7f720e8f29968ad0a483441553c0c885b87b3df84568.
• —Agent: promoted v6-u96 dual-gate-passed checkpoint, SHA-256 d81464982c3ebc0d72769a87e079068bf535d6ca2010185dc1bb03ce264b8f5b.
• —Conversion: bitwise-verified tensor-by-tensor against the source state dicts.
• —Port qualification: GGML runner + registered llama.cpp arch vs GPU reference: 32/32 top-1, KL 0.0007 (f32).

E2-u512 (E2 chat-agent arc, segment-4 checkpoint) — RELEASED 2026-09-23, mid-arc candidate, NOT PROMOTED

• —File: e97-4b-e2-u512-q8_0.gguf (4.87 GB; q8_0, KL-qualified quantization class per the runner-vs-GPU gate)
• —Checkpoint: 5ea4e078329f0d960b7311b09bccf26def3913ef818dfb0d109d91cecd08a7e1 (E2 arc segment 4, cumulative u512-equivalent, chained from E1-u256 853a3b95...)
• —Status: mid-arc candidate; the full dual gate evaluation was in flight at publication time. Prior arc readings: execution 65/96 at u256 (first crossing of the 64 floor in programme history, bridge control 67/96 in-run), retention 1.516-1.527 (limit 1.72), long-document NLL 2.555 (base 2.492, bridge 2.610 — the anchor cohorts reversing drift monotonically), chat probe 0/3 (the diagnosed seam-dose gap; the 5,200-record verified hybrid family targets it in E3), Stage-B oscillating 9-11/14 (floor 12; the late-arc oscillation finding points at the LR schedule, untested).
• —Diet: E1's families rebalanced (OH-raw 10%, tool-talk 5%) + long-document anchor cohorts at 19.6% (fragment render: arbitrary-entry window cuts over the concatenated long-document stream, separator-boundary hidden-state resets; monotone doc-NLL gains across all four junctures).
• —Honest caveats: un-promoted; the chat experience remains weak (phantom tool calls / malformed frames on plain-chat prompts); greedy decoding repetition-collapses (sample: temp ~0.7); trained in the canonical Pi-native episode codec, not ChatML.