alibiserikbay/JevK5-2B

JevK5-2B v0.2: the small JevK5

JevK5-2B is the 2-billion-parameter version of JevK5, an Apache-2.0 open alternative to TypeSafe's Jev for typed decisions. It reads a document and a yes/no (noul), choice, or score question and returns a probability for every option in one forward pass, with nothing generated. It is not affiliated with TypeSafe AI.

It was trained on exactly the same data, the same way, as JevK5 v0.2 (4B), on the smaller Qwen3.5-2B. It keeps most of JevK5's accuracy at half the memory: good at reading a document and deciding, weaker on arithmetic, multi-step reasoning, and general knowledge.

• —Base: Qwen3.5-2B, with a LoRA (rank 16, attention and linear-attention projections) merged into the weights
• —Readout: a softmax over the answer letters' next-token logits, divided by one calibration temperature (jevk5_config.json, T = 1.42)
• —Memory: 3.5 GB of bf16 weights, about 3.8 GB at peak on a 4,000-token document; a 2.0 GB GGUF (Q8_0) is in JevK5-GGUF
• —Speed: about 9 ms per decision on an H100 with the JevK5 runtime
• —License: Apache-2.0

Results

Held-out teacher questions. 362 questions from three business domains that training never saw (residential leases, public-sector permits, manufacturing QC). This is the set used to choose between models and to fit the temperature.

Each question family has only 21-42 items, so the per-family rows are rough.

JevBench's 231 public decisions, through JevBench's own runner and scoring. Reported only; never used to choose anything. These are our runs, not official results: JevK5-2B has not been submitted to JevBench.

How it was trained

The same data and recipe as JevK5 v0.2. Qwen3.6-27B, with thinking on, wrote realistic documents with hard typed questions across 17 business domains and answered every question twice, independently; a question was kept only when both answers matched the intended one. Training used 3,272 of those questions plus as many human-labelled items from MMLU-Pro, WANLI, MultiNLI, BoolQ, banking77, ARC and CommonsenseQA: cross-entropy on the option-letter logits, 2 epochs, learning rate 3e-5. The temperature was fitted on the held-out teacher questions above.

We also tried, at 2B, distilling JevK5's own option probabilities, three option orders per question, 3,400 extra replay questions, and a higher learning rate with a larger adapter. None improved the held-out teacher questions, so this is the plain recipe.

No JevBench item, public or held out, and no output of Jev was used for training, tuning or selection. One correction, shared with JevK5: our hand-written calibration set echoed a public JevBench instruction and one public item's rule wording. Both are rewritten; see CHANGELOG.md in the repository.

Known weak spots

• —Dates, amounts and multi-step lookups: the largest gap to the 4B model.
• —The standard tier drops to 0.806 from the 4B's 0.958.
• —Not measured on JevBench's sealed decisions, where JevK5 (4B) drops to 33.1% and every one-pass model struggles. Expect the same or worse here.
• —English only. Inputs over 16,384 tokens are refused, not cut.

Use

With the JevK5 runtime on an NVIDIA GPU:

pip install "jevk5[fast] @ git+https://github.com/allebee/jevk5@v0.2.0"

from jevk5 import JevK5

model = JevK5("alibiserikbay/JevK5-2B")
model.decide(
    "I was billed twice for order #4411. Please refund the duplicate charge today.",
    {"type": "choice", "instructions": "Which team should handle this?",
     "criteria": {"billing": "Payments and refunds", "tech": "Bugs", "sales": "New purchases"}},
)
# {'type': 'choice', 'choice': 'billing', 'confidence': 0.99, ...}

On other GPUs, on a Mac, or on a CPU, use the GGUF build with llama.cpp: see JevK5-GGUF.

Credits

Qwen3.5-2B and Qwen3.6-27B by the Qwen team (Apache-2.0). The one-pass readout and prompt come from SemIf by TheoLeeCJ (MIT). Evaluated with JevBench (github.com/fstandhartinger/jevbench, MIT). Not affiliated with TypeSafe AI or Jev.