kthierbach/unlimited-ocr-max

Unlimited-OCR, served through MAX

`baidu/Unlimited-OCR` as one repository max serve opens directly: baidu's weights unchanged (model.safetensors, byte-identical to upstream at revision 07dea832e22aefee32ad281d4b80551282e1c168, sha256 2bc48a7a110061ea58fff65d3169367eebe3aee371ca6968dc2219c1b2855fc6), the tokenizer files as published, and the checkpoint's config.json with two keys removed (auto_map, model_type) so MAX can load it without trust_remote_code. The runtime is a MAX custom architecture with three Mojo custom ops, served from https://github.com/kthr/unlimited-ocr-max as an OpenAI-compatible endpoint on the GPU or the CPU. The repository also carries `model-int8.safetensors`, this port's weight-only int8 quantisation of the same checkpoint (symmetric per-group G=128, routed experts only).

Install and serve

uv tool install unlimited-ocr-max
unlimited-ocr-max serve --devices gpu                 # bf16: downloads model.safetensors once (6.2 GiB)
unlimited-ocr-max serve --devices gpu --weights int8  # int8: 4.0 GiB download, faster decode

pip install unlimited-ocr-max into a venv is the equivalent. No extra index: the exact max[all]==26.6.0 pin is the stable release, on PyPI with the mojo it depends on. --devices cpu is supported (slow).

--revision defaults to `v0.3.0`, the tag of this repository the package version was validated against — keep it: a fixed package version then resolves fixed weights, config and tokenizer, so a later push here cannot silently falsify the fidelity claims below. Offline route: hf download kthierbach/unlimited-ocr-max --revision v0.3.0 --local-dir <dir> once, then serve --model <dir>. (A local copy of baidu/Unlimited-OCR itself cannot be served this way — its config.json demands remote code; use this repository.) Requests go to http://127.0.0.1:8010/v1/chat/completions as one image plus <|grounding|>Convert the document to markdown., temperature 0; the no-repeat-n-gram guard defaults to the model's own 35.

Hardware and prerequisites

Nothing in the port is Apple-specific — it is a plain MAX custom architecture plus Mojo kernels — and max==26.6.0 targets three accelerator backends: Metal, ROCm/HIP and CUDA. (AWS Trainium and Google TPU are announced for the Modular Platform; they are not backends of this release, so they are not claimed here.) Per target, exactly what has been established:

• —Python 3.12 or 3.13 (max itself supports 3.10–3.14; this package pins the narrower range). Hub access on first run; weights are cached after.
• —24 GB unified memory recommended on Apple silicon: startup compilation can peak near 16–20 GiB resident on a cold compile cache, while steady serving sits at the residency in the table below. One model process at a time.

Fidelity and performance

Measured 2026-09-18 on max==26.6.0 (stable) / Mojo 1.1.0, Apple M4 24 GB, macOS 26.5.2, over the twelve-page benchmark corpus, temperature 0, default guard, one draw per arm; served code byte-identical to v0.3.0. The reference is the vendored PyTorch implementation (fp32, CPU, transformers 4.46.3).

int8's divergence is 27 edited characters of 24 019 — every one a bounding-box digit (±1–2 px), none a word, none document structure — for 1.8× bf16's decode rate and a 4.0 GiB download. bf16's low steady residency comes from v0.3.0's device-weight registry (one shared device copy of the language weights; the earlier ~10 s per-request graph reload is gone below the scheduler's measurement tick). int8 deliberately does not share weights — a served gate showed the registry corrupts int8 output — so it keeps host copies and the higher steady residency. Method, per-page rows and the void rules are in the code repository's BENCHMARKS.md and changelog.

Not supported

• —gundam mode (dynamic tiling) is not served — base mode only.
• —Batch size > 1 (the prefill graph's sequence length is static).
• —Multi-GPU.

References

The model is described in Unlimited OCR Works (Yin et al., 2026), arXiv:2606.23050. This repository changes how the model is served, not the model (int8 is the one exception, and its divergence is quantified above). Model-level behaviour, limitations and biases are those of baidu/Unlimited-OCR.

@misc{yin2026unlimitedocrworks,
      title={Unlimited OCR Works},
      author={Youyang Yin and Huanhuan Liu and YY and Qunyi Xie and Chaorun Liu and Shiqi Yang and Shaohua Wang and Zhanlong Liu and Hao Zou and Jinyue Chen and Shu Wei and Jingjing Wu and Mingxin Huang and Zhen Wu and Guibin Wang and Tengyu Du and Lei Jia},
      year={2026},
      eprint={2606.23050},
      archivePrefix={arXiv}
}

Authorship

This port, its serve configuration and this card were written with substantial AI assistance (Claude, via Claude Code); Konstantin Thierbach reviewed them and is accountable for their contents. Assisted-by: AI.

License

MIT — see `LICENSE` in this repository: this port's MIT notice and baidu's MIT notice verbatim. The bf16 weights, tokenizer files and config.json are baidu's, from baidu/Unlimited-OCR (MIT, Copyright (c) 2026 Baidu), redistributed under that license; the bf16 weights are unchanged, config.json has two keys removed, and model-int8.safetensors is derived from those weights by this port's quantiser.