Modelpublic

AnkitAI/tinyjev-0.6b

sourceHugging Facemitupdated 13h agoView on Hugging Face

0likes

Model Card

<p>Typed decisions, on your laptop, in one forward pass.</p>

<p> <a href="https://github.com/ankit-aglawe/tinyjev">GitHub</a> · <a href="https://pypi.org/project/tinyjev/">PyPI</a> · <a href="https://github.com/ankit-aglawe/tinyjev/tree/main/examples">Examples</a> </p>

<p> <b>English</b> · <a href="https://github.com/ankit-aglawe/tinyjev/blob/main/README.zh-CN.md">简体中文</a> · <a href="https://github.com/ankit-aglawe/tinyjev/blob/main/README.ja.md">日本語</a> · <a href="https://github.com/ankit-aglawe/tinyjev/blob/main/README.ko.md">한국어</a> </p>

</div>

Send this model some state, a ticket or a record or a log line, plus questions with the answers you will accept. It returns a probability for every option you offered. It cannot answer with anything else, because it never generates text; it scores the options you gave it and stops.

—Choice picks one option from a list, with a probability for each.
—Noul measures whether a statement is true.
—Score places state on an ordered scale.
—Confidence is calibrated, so a threshold means something.

596M parameters, about 1.2 GB. MLX on Apple Silicon, PyTorch everywhere else, fully offline.

Watch it decide

Eight real support tickets, three questions each in a single forward pass, about 110 ms per ticket on a base M1. Every number in that recording came from a live run.

Use it

bash

pip install 'tinyjev[mlx]'     # Apple Silicon
pip install 'tinyjev[torch]'   # everything else

python

import tinyjev
agent = tinyjev.load("tinyjev-0.6b")

agent.predict({
    "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
    "questions": {
        "team":     {"type": "choice", "instructions": "Which team should handle this?",
                     "criteria": {"returns": "Exchanges, refunds, wrong or damaged items",
                                  "shipping": "Delivery status, delays, lost packages",
                                  "billing":  "Charges, invoices, payment problems"}},
        "escalate": {"type": "noul",   "instructions": "Does this need urgent human attention?"},
        "anger":    {"type": "score",  "instructions": "How angry is the customer?",
                     "criteria": ["calm", "frustrated", "very angry"]},
    }})

On Apple Silicon you can quantize as it loads. Eight bits is free: half the memory, slightly faster, and it scored identically to full precision on our held-out set.

python

agent = tinyjev.load("tinyjev-0.6b", quantize=8)

Serve it over HTTP, speaking the System One request shape:

bash

tinyjev serve tinyjev-0.6b        # POST /v1/systemone on 127.0.0.1:8077

What is in this repo

AutoModel.from_pretrained("AnkitAI/tinyjev-0.6b") loads the backbone on its own, a standard Qwen3Model in fp16. The decision head lives in head.safetensors, and tinyjev is what turns hidden states into calibrated answers.

How it was built, and how it scores

Qwen3-0.6B-Base with a pointer head, LoRA r16 at lr 5e-5 merged back into the base, trained on the public jaredpalmer/kev-suites decision-v7 split. No held-out transfer source was used in training. A fitted temperature of 1.46 is applied at inference.

	transfer-v4 dev	transfer-v4 test, read once	ECE on test
tinyjev-0.6b	0.625	0.663	0.082
Same-size public anchor	0.620	0.642	—

Scored with the upstream harness on its frozen held-out suite. This matches the same-size public anchor and edges ahead on the locked test with lower calibration error. It is not 4B-class, and it is not meant to be. Full fine-tuning, distillation from a 4B teacher, and a 149M encoder were all tried and all lost to the configuration above.

tinyjev-0.6b is done and published. Next is a smaller one, around 0.15B.

Credits

Built on Qwen3-0.6B-Base (Apache-2.0). The training data, evaluation suites and the pointer-head design come from Kev by Jared Palmer (Apache-2.0). The typed-decision interface follows TypeSafe's Jev. MIT licensed.