nutrientdocs/doc-split-v1

1---2license: apache-2.03pipeline_tag: image-text-to-text4language: [multilingual]5tags:6  - page-stream-segmentation7  - document-boundary-detection8  - document-ai9  - document-splitting10  - open-weights11datasets:12  - nutrientdocs/doc-split-benchmark13---14 15# doc-split-v1 — open-weight16 17**Where does one document end and the next begin?** An open-weight page-stream-segmentation model you can18download and run: it splits a stream of pages (a scanned batch / merged PDF) back into its constituent19documents.20 21The lightweight, open sibling of the commercial flagship22[`doc-split-v2`](https://huggingface.co/nutrientdocs/doc-split-v2) — compact, ~4.5× faster, near-flagship23accuracy on our data and multilingual out of the box. Shipped as **ONNX** — runs with `onnxruntime`, no24framework or modelling code to install.25 26- 🎯 **Try it:** [doc-split-demo](https://huggingface.co/spaces/nutrientdocs/doc-split-demo?model=v1)27- 🏆 **Leaderboard:** [doc-split-leaderboard](https://huggingface.co/spaces/nutrientdocs/doc-split-leaderboard)28- 📊 **Benchmark:** [doc-split-benchmark](https://huggingface.co/datasets/nutrientdocs/doc-split-benchmark)29- 🔒 **Higher accuracy?** [doc-split-v2](https://huggingface.co/nutrientdocs/doc-split-v2) (commercial)30 31## Results — boundary F1 (κ)32 33Per-page boundary detection, page 0 forced. **This model** vs the private doc-split-v2, the strongest cloud VLM,34and prior work.35 36| Cut | **doc-split-v1** | doc-split-v2 | best cloud VLM | OpenPSS specialist |37|---|---|---|---|---|38| OpenPSS-short (sparse) | **0.585** (.53) | 0.619 | 0.598 (gemini-flash) | 0.76 |39| OpenPSS-long | **0.859** (.82) | 0.886 | 0.244 (gemini-flash) | 0.83 |40| our-200 (synthetic) | **0.936** (.78) | 0.934 | 0.942 (gpt-sol) | — |41| TABME++ test | **0.704** (.56) | 0.901 | — | — |42| Tobacco800 test | **0.820** (.60) | 0.957 | — | — |43| val (real-doc) | **0.918** (.86) | 0.908 | — | — |44 45Beats every evaluated cloud VLM on OpenPSS-**long** (0.859 vs 0.244) at a fraction of the cost, and holds up46on our data. TABME++/Tobacco800 are zero-shot for this model (in-domain for doc-split-v2).47 48## What's in this repo49 50Runs entirely under `onnxruntime` — nothing else to install.51 52- `image_model.onnx`, `text_model.onnx` — the image and text towers (per-page embeddings).53- `head.onnx` — the boundary head (per-page boundary score).54- `crf.json` — smoothing parameters for the per-page confidence.55- `tokenizer.json` (+ config) — the bundled text tokenizer.56 57## Usage (ONNX)58 59```python60# pip install onnxruntime transformers numpy huggingface_hub61import numpy as np, onnxruntime as ort, json62from transformers import AutoTokenizer63from huggingface_hub import snapshot_download64 65d = snapshot_download("nutrientdocs/doc-split-v1")66img  = ort.InferenceSession(f"{d}/image_model.onnx", providers=["CPUExecutionProvider"])67text = ort.InferenceSession(f"{d}/text_model.onnx",  providers=["CPUExecutionProvider"])68head = ort.InferenceSession(f"{d}/head.onnx",        providers=["CPUExecutionProvider"])69tok  = AutoTokenizer.from_pretrained(d); crf = json.load(open(f"{d}/crf.json"))70 71def _lse(x, ax):72    m = x.max(ax, keepdims=True); return (m + np.log(np.exp(x - m).sum(ax, keepdims=True))).squeeze(ax)73 74def marginals(bl, crf):                    # per-page confidence via forward-backward over a 2-tag chain75    T = np.asarray(crf["trans"]); s = np.asarray(crf["start"]); e_ = np.asarray(crf["end"])76    N = len(bl); e = np.stack([np.zeros(N), bl], 1); a = np.zeros((N, 2)); a[0] = s + e[0]77    for t in range(1, N): a[t] = _lse(a[t-1][:, None] + T, 0) + e[t]78    b = np.zeros((N, 2)); b[N-1] = e_79    for t in range(N-2, -1, -1): b[t] = _lse(T + (e[t+1] + b[t+1])[None, :], 1)80    m = a + b; m -= m.max(1, keepdims=True); p = np.exp(m); return (p / p.sum(1, keepdims=True))[:, 1]81 82def split(pages, tau=0.5):                 # pages: list of (PIL image, ocr_text or "")83    arr = np.stack([(np.asarray(im.convert("RGB").resize((512, 512)), np.float32)/255 - .5)/.584                    for im, _ in pages]).transpose(0, 3, 1, 2).astype(np.float32)85    vi = img.run(["image_embed"], {"pixel_values": arr})[0]86    b = tok(["query: "+(t or " ") for _, t in pages], padding=True, truncation=True,87            max_length=512, return_tensors="np")88    vt = text.run(["text_embed"], {"input_ids": b["input_ids"].astype(np.int64),89                                   "attention_mask": b["attention_mask"].astype(np.int64)})[0]90    g = np.array([1. if (t and t.strip()) else 0. for _, t in pages], np.float32); N = len(pages)91    vt = vt * g[:, None]                    # OCR gate: text ignored on pages with no text layer92    bl = head.run(["boundary_logit"], {"v_img": vi[None], "v_txt": vt[None],93                   "gate": g[None], "mask": np.ones((1, N), np.float32)})[0][0]94    bl[0] = 30.0                           # force page 0 to start a document95    conf = marginals(bl, crf)              # per-page confidence in [0,1]96    return [1 if (i == 0 or conf[i] >= tau) else 0 for i in range(N)]   # 1 = this page starts a new document97```98 99## Intended use & limits100 101**Use it for:** splitting merged/batch-scanned PDFs into documents; routing; pre-processing for102classification/extraction. **Limits:** boundary detection only (does not classify document *type*); the103sparse low-boundary regime (OpenPSS-short) is hardest; OCR text helps on text-heavy pages.104 105## License106 107Apache-2.0.108 109## Calibrated confidence110 111The raw boundary score is over-confident (a raw 0.85 is really ~63% likely a true boundary). We ship a112**beta calibration** (fit on held-out data) so the reported confidence is honest and usable as a threshold:113 114```115p_calibrated = sigmoid(a·ln(p) + b·ln(1-p) + c),  (a, b, c) = (0.516, -0.402, -0.155)116```117ECE 0.044 → 0.012. The demo applies this and lets you set a minimum-confidence threshold on the calibrated value.118 119## About the author120 121<a href="https://nutrient.io/">122  <img src="https://avatars2.githubusercontent.com/u/1527679?v=3&s=200" height="80" />123</a>124 125This project is maintained and funded by [Nutrient](https://nutrient.io/) - The deterministic document infrastructure enterprises run their highest-stakes workflows on: replayable output, clear exceptions, and full audit trails on the messy, regulated documents where AI alone breaks.126