distil-labs/distil-qwen3.5-4b-invoice-decision-gguf

Distil-Qwen3.5-4B-Invoice-Decision (GGUF)

A fine-tuned Qwen3.5-4B that reads a vendor invoice, the purchase order and the goods receipt, reasons through four checks, and answers pay or hold with one of five decisions. It is step 2a of the invoice processing pipeline: the decision-only variant, built to compare a model that reasons with Jev on an output Jev can produce. The pipeline itself uses the grounded decision model.

Trained on the distil labs platform from 40 seed examples: a teacher model (GLM 5.3 with reasoning) generated 4,156 synthetic training examples, and the student was fine-tuned on them. This repo holds the Q80 GGUF build (`distil-qwen3.5-4b-invoice-decision-q80.gguf`). Safetensors weights: distil-labs/distil-qwen3.5-4b-invoice-decision.

Results

The test set has 32 invoices to approve (20 of them near misses, such as a price 1.8% above the PO price), 60 that fail one check and 8 that fail two. Models that answer in one pass score 75 to 84; this model reasons first and catches all 16 wrong totals. Both of its errors are slips in the running sum on line totals in the thousands.

How it was scored, all baselines, and the raw outputs: https://github.com/distil-labs/invoice-processing-pipeline.

How to use it

The model was trained to reason before it answers, in a short fixed format (the checks in policy order, one line per invoice line, a running sum, stop at the first failure; about 160 tokens). Serve it with thinking on: chat_template_kwargs: {"enable_thinking": true}. The answer follows the reasoning.

Serve it behind an OpenAI-compatible endpoint:

hf download distil-labs/distil-qwen3.5-4b-invoice-decision-gguf distil-qwen3.5-4b-invoice-decision-q8_0.gguf --local-dir models
llama-server -m models/distil-qwen3.5-4b-invoice-decision-q8_0.gguf --port 8001 --jinja -c 16384 -np 4

--jinja is required: the chat template carries the thinking switch.

Call it with the system prompt it was trained with, at temperature 0:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8001/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="model",
    messages=[{"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": text}],
    temperature=0,
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)

Example answer:

{"decision": "hold_price"}

The model degrades outside its training setup, so keep the system prompt exactly as below. The pipeline repo loads it from training/decider/job_description.json and shows the input format.

<details> <summary>System prompt</summary>

You check vendor invoices for the accounts payable team of Northwind before they are paid.

You receive three documents: the invoice message as it arrived by email, the purchase order (PO) from the ERP, and the goods receipt from the ERP. Invoice lines can be in a different order than the PO lines and can use the vendor's own item names or abbreviations, so match each invoice line to its PO line by item. Amounts in the message that are not part of this invoice (a previous balance, a payment received, a quote) are ignored.

Run these checks in order and stop at the first one that fails:
1. The PO number referenced on the invoice is exactly the number of the PO. If not: hold_no_po.
2. On every invoice line, the invoiced quantity is not more than the received quantity of that item. If not: hold_quantity.
3. On every invoice line, the invoiced unit price is not more than 2% above the PO unit price of that item. A lower price is fine. If not: hold_price.
4. The total stated on the invoice equals the sum of the invoice line totals, plus the freight charge only if the PO says freight may be added by the vendor. If the PO says freight is not allowed, a freight charge must not be part of the total. If not: hold_total.
If every check passes: approve.

Answer with a JSON object and nothing else: {"decision": "<label>"}, where <label> is one of approve, hold_no_po, hold_quantity, hold_price, hold_total.

</details>

Training

Seed data, test set, job description and config: `training/decider`.

Limits

The data is synthetic, written for this demo around a fictional company, in English, with amounts in one currency. The model is trained for this one task and this one policy; it is not a general assistant.

Links

distil labs · GitHub · Hugging Face · LinkedIn · Slack · X