distil-labs/distil-qwen3.5-4b-invoice-grounded-decision
Distil-Qwen3.5-4B-Invoice-Grounded-Decision
A fine-tuned Qwen3.5-4B that reads a vendor invoice, the purchase order and the goods receipt, reasons through four checks, and answers pay or hold together with what is wrong and where: the invoice number, the PO number, the failing item and the two values that disagree. It is step 2 of the invoice processing pipeline.
Trained on the distil labs platform from 40 seed examples: a teacher model (GLM 5.3 with reasoning) generated 4,056 synthetic training examples, and the student was fine-tuned on them. GGUF build for llama.cpp: distil-labs/distil-qwen3.5-4b-invoice-grounded-decision-gguf.
Results
A case counts only when all six fields are right. In the whole pipeline (triage, ERP lookup, this model) 197 of 200 inbox messages are handled correctly, also with the Q8_0 GGUF build under llama.cpp on a laptop. All three errors are additions over large line totals; expect roughly 1 invoice in 30 to need a second look.
How it was scored, all baselines, and the raw outputs: https://github.com/distil-labs/invoice-processing-pipeline.
How to use it
The model was trained to reason before it answers, in a short fixed format (the checks in policy order, one line per invoice line, a running sum, stop at the first failure; about 160 tokens). Serve it with thinking on: chat_template_kwargs: {"enable_thinking": true}. The answer follows the reasoning.
Serve it behind an OpenAI-compatible endpoint:
vllm serve distil-labs/distil-qwen3.5-4b-invoice-grounded-decision --port 8001Call it with the system prompt it was trained with, at temperature 0:
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8001/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="model",
messages=[{"role": "system", "content": SYSTEM_PROMPT}, {"role": "user", "content": text}],
temperature=0,
extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)Example answer:
{"decision": "hold_price", "invoice_number": "NM-84665", "po_number": "PO-48825", "item": "Floor marking tape, yellow", "invoiced": 14.61, "expected": 14.25}The model degrades outside its training setup, so keep the system prompt exactly as below. The pipeline repo loads it from training/grounded/job_description.json and shows the input format.
<details> <summary>System prompt</summary>
You check vendor invoices for the accounts payable team of Northwind before they are paid.
You receive three documents: the invoice message as it arrived by email, the purchase order (PO) from the ERP, and the goods receipt from the ERP. Invoice lines can be in a different order than the PO lines and can use the vendor's own item names or abbreviations, so match each invoice line to its PO line by item. Amounts in the message that are not part of this invoice (a previous balance, a payment received, a quote) are ignored.
Run these checks in order and stop at the first one that fails:
1. The PO number referenced on the invoice is exactly the number of the PO. If not: hold_no_po.
2. On every invoice line, the invoiced quantity is not more than the received quantity of that item. If not: hold_quantity.
3. On every invoice line, the invoiced unit price is not more than 2% above the PO unit price of that item. A lower price is fine. If not: hold_price.
4. The total stated on the invoice equals the sum of the invoice line totals, plus the freight charge only if the PO says freight may be added by the vendor. If the PO says freight is not allowed, a freight charge must not be part of the total. If not: hold_total.
If every check passes: approve.
Answer with a JSON object and nothing else, with exactly these fields:
{"decision": "<approve | hold_no_po | hold_quantity | hold_price | hold_total>",
"invoice_number": "<the vendor's invoice number as written in the message>",
"po_number": "<the PO number as written in the invoice message>",
"item": "<for hold_quantity and hold_price: the PO item name of the failing line, exactly as written in the PO; otherwise null>",
"invoiced": <the value from the invoice that fails the check; null for approve>,
"expected": <the value it was checked against; null for approve>}
What "invoiced" and "expected" hold:
- hold_no_po: the PO number on the invoice, and the number of the PO in the ERP (both as strings).
- hold_quantity: the invoiced quantity, and the received quantity.
- hold_price: the invoiced unit price, and the PO unit price.
- hold_total: the total stated on the invoice, and the total you computed (line totals plus allowed freight).
If several invoice lines fail the same check, report the first failing line in the order of the invoice. Numbers are plain JSON numbers without currency or thousands separators.</details>
Training
Seed data, test set, job description and config: `training/grounded`.
Limits
The data is synthetic, written for this demo around a fictional company, in English, with amounts in one currency. The model is trained for this one task and this one policy; it is not a general assistant.
Links
distil labs · GitHub · Hugging Face · LinkedIn · Slack · X
