quaky-duck/kev-8b-decision-model

kev-8b — a decision model

`jaredpalmer/kev-8b` is a LoRA adapter (r=16) plus a pointer readout head on top of a frozen `Qwen/Qwen3-8B-Base` backbone. It reads a document once and answers many typed questions about it in parallel, in a single prefill pass with no decoding.

• —The state and every question are packed into one token sequence.
• —A block-causal mask lets each question attend to the state, but never to a sibling question.
• —A pointer head scores each option's </opt> hidden state against the question's <decide> hidden state and applies a softmax. Those probabilities are the output — trained with cross-entropy against labelled outcomes, not generated as text.

Three question types: noul (yes/no), choice (2–255 named options), score (ordered levels). The API mirrors TypeSafe's POST /v1/systemone contract.

The most accurate kev. 0.870 in-distribution and 0.780 out-of-domain on the project's locked test (the hosted Jev reference scores 0.845 / 0.857 on the same items).

How the app runs the checkpoint

• —bf16, LoRA merged. The model card gives ~33 GB for fp32 and ~17 GB for bf16, so bf16 it is. kev.evaluate.load merges the adapter into the base weights in fp32 before casting, and measures that as both ~15% faster and closer to the fp32 reference than serving the adapter unmerged in bf16 (max |dp| 0.017 vs 0.029, 0 vs 1 argmax flips). Following that literally would mean holding an fp32 Qwen3-8B (~33 GB) in the main process, so kev_core/loader.merge_lora_fp32 merges module by module and keeps only the sum in fp32. Qwen3-8B-Base ships bf16 on the Hub, so W.float() is lossless and this computes exactly the same round(W + B@A × scaling) at bf16 peak memory.
• —Assembled on CPU, one eager `.to("cuda")` at the end. On ZeroGPU a module-scope "cuda" tensor is a bookkeeping entry rather than a device buffer, so the merge arithmetic has to happen on real CPU tensors — and peft would otherwise ask safetensors to materialise the adapter straight onto a GPU that isn't there.
• —Packed-sequence cap of 8,192 tokens. The branch mask is a dense [1, 1, L, L] tensor, so total packed length is what actually costs memory. The checkpoint was trained at 384 state / 1,024 branch tokens.

Two optional checks in the UI:

• —Branch isolation re-asks every question in its own forward pass and reports the largest probability difference against the packed answers. It should be ~1e-6 — a question cannot see a sibling.
• —Option order re-asks the first choice question under shuffled option orders. Kev-8B's measured option-order flip rate is 0.00.

The example presets are the ones shipped with the project's own playground (playground/src/lib/kev.ts), including the Isolation probe and Boundary forgery experiments from the architecture write-up.

Limitations

Straight from the model card — worth reading before trusting an answer:

• —Out of domain it trails Jev by ~6 points, concentrated in knowledge (MMLU 0.70 vs 0.90), date arithmetic (0.60 vs 0.93) and one held-out policy structure (0.59 vs 0.78).
• —Calibration is in-distribution only. Out-of-domain probabilities are usable but not calibrated (ECE ~0.1), with ~9.9% confident errors (p ≥ 0.9 and wrong).
• —Product-shaped questions with no training analogue are not guaranteed. Measure on your own data.

Credits & licence

Model, inference code (kev_core/model.py, kev_core/api.py, vendored verbatim from the project) and the playground presets are by Jared Palmer, Apache-2.0 (source). Architecture reconstruction by Archer Hume; API contract from TypeSafe. The base model is distributed under the Qwen licence.