nutrientdocs/document-classification-v2
document-classification-v2 · commercial
Classify any document image against labels you choose at runtime. document-classification-v2 is an open-vocabulary, zero-shot document classifier: supply one label + a short description per class at inference, and it scores a document page against any such set — invoices, letters, memos, forms, scientific articles, resumes, and whatever label you type next. No fixed class list, no per-class training.
It is the document specialist in a field of generalists. General open-vocab models were trained on web photos; this one is built for document pages — and it shows on the leaderboard. Weights are commercial (not downloadable here); this page is a spec + scorecard.
- 🎯 Try it: document-classification-demo
- 🏆 Leaderboard: document-classification-leaderboard
- 📊 Benchmark: document-classification-benchmark
Results
Macro-F1, zero-shot, on the held-out benchmark — higher is better. The flagship vs the previously-shipped in-house model and a cloud frontier VLM reference.
<!-- RESULTS-TABLE:START — filled from leaderboard/leaderboard.json by the scoring run --> | Benchmark (macro-F1) | document-classification-v2 | best generalist | best cloud VLM | | --- | ---: | ---: | ---: | | DocLayNet (page types) | 0.97 | 0.63 | 0.83 | | Forms | 1.00 | 0.11 | 1.00 | | Tobacco (doc types) | 0.74 | 0.43 | 0.85 | | OOD (unseen doc types) | 0.95 | — | — | | OOV (synonym wording) | 0.83 | — | — |
Latency (fused image+text, A40): ~4.9 pages/s on text-heavy pages; the image branch alone runs ~8 docs/s when no OCR is present. The open-weight `v1` runs ~5.7 pages/s fused on the same GPU. <!-- RESULTS-TABLE:END -->
Every model — ours and cloud — is scored by the same open macro-F1 scorer; full per-model ranking on the leaderboard.
- Matches/leads the cloud on visual document-type tracks (DocLayNet, Forms) at zero per-request API cost — and runs on a single GPU.
- Trails on Tobacco (0.74 vs 0.85): that track rewards reading fine header text (memo vs letter vs email) — a large VLM reads it; an embedding model can't.
- OOD robustness: on document types absent from training (invoices, handwriting, charts, tables), 0.95 macro-F1. Open-vocab (OOV): under never-seen synonym label wording, 0.83 — matches the concept, not your exact string. (Cloud VLMs train on ~all data, so these held-out axes aren't reported for them.)
Intended use & limits
- Use it for: zero-shot classification of document page images in a free-label setting — the caller supplies the candidate labels (and optional descriptions). Multi-page documents supported; optional page OCR sharpens fine-grained form / tax-code distinctions.
- Limits: optimized for document imagery; English label strings are the primary target. Scores are per-label match probabilities (independent per label), not a softmax across the set.
License & data
The model weights are offered under a commercial Nutrient license — deployed on-prem, so your documents never leave your infrastructure. The training set is not redistributed. Evaluation runs on the held-out document-classification-benchmark.
### 📩 Get access document-classification-v2 is commercial and its weights are not downloadable here. To run it on-prem — open-vocabulary, calibrated, private — contact Nutrient: [nutrient.io/contact-sales](https://www.nutrient.io/contact-sales/).About the author
<a href="https://nutrient.io/"> <img src="https://avatars2.githubusercontent.com/u/1527679?v=3&s=200" height="80" /> </a>
This project is maintained and funded by Nutrient - The deterministic document infrastructure enterprises run their highest-stakes workflows on: replayable output, clear exceptions, and full audit trails on the messy, regulated documents where AI alone breaks.