dougvk/Unlimited-OCR-RDNA4

1---2pipeline_tag: image-text-to-text3license: mit4tags:5  - rocm6  - amd7  - rdna48  - gfx12019  - ocr10  - runtime11---12 13# Unlimited-OCR RDNA 414 15Single-GPU [Baidu Unlimited-OCR](https://huggingface.co/baidu/Unlimited-OCR) inference for AMD RDNA 416(`gfx1200` / `gfx1201`) using native ROCm PyTorch, verified end to end on `gfx1201`.17 18This project does **not** redistribute or rename Baidu's weights. It provides a reproducible runtime around the original19BF16 checkpoint: an isolated AMD wheel bootstrap, pinned model preparation, audited custom-code patching, single-GPU20selection, local-only model resolution during inference, atomic output, bounded PDF handling, diagnostics, and21validation evidence.22 23> Status: beta. Verified on 31 July 2026 with an AMD Radeon RX 9070 XT (`gfx1201`). This is not an AMD or Baidu project.24 25## Verified configuration26 27| Component | Verified value |28|---|---|29| GPU | AMD Radeon RX 9070 XT, 16 GB |30| GPU allocation | One isolated GPU |31| OS | Ubuntu 24.04.4 |32| Kernel | 6.17.0-40-generic |33| Host ROCm | 7.2.1 |34| PyTorch | 2.9.1 + ROCm 7.2.1 |35| Transformers | 4.57.1 |36| Matplotlib | 3.10.8 |37| Model revision | `07dea832e22aefee32ad281d4b80551282e1c168` |38| Weight precision | Original BF16; no quantization |39| Peak allocated VRAM | 8.25 GiB on the 1600×2000 validation page |40 41The initial validation recovered headings, reading order, a structured table, euro amounts, a formula, and an exact42checksum. Repeated greedy generation was byte-identical after deterministic tag cleanup. See43[docs/VALIDATION.md](docs/VALIDATION.md) for the evidence and its limits.44 45## Quick start46 47The bootstrap downloads checksum-pinned AMD ROCm wheels into your XDG cache, installs every other runtime/build48dependency from `requirements/bootstrap.lock` with hashes required, builds this checkout without build isolation, and49creates a repository-local virtual environment. It does not use `sudo`, alter `/opt/rocm`, or touch the system Python.50 51```bash52git clone https://huggingface.co/dougvk/Unlimited-OCR-RDNA453cd Unlimited-OCR-RDNA454 55./scripts/bootstrap-rocm.sh56.venv/bin/unlimited-ocr-rdna4 prepare57.venv/bin/unlimited-ocr-rdna4 run --input page.png58```59 60The model download is approximately 6.78 GB. The verified AMD wheel set is approximately 1.9 GB before installation.61 62PDFs are rendered and parsed page by page:63 64```bash65.venv/bin/unlimited-ocr-rdna4 run \66  --input document.pdf \67  --output document.md \68  --max-pages 10 \69  --dpi 20070```71 72## Commands73 74### `prepare`75 76Downloads the pinned original checkpoint, fully hashes the 6.67 GB safetensors file and every behavior-defining model,77configuration, tokenizer, and index file, preserves Baidu's original model source, and applies a narrowly scoped audited78patch.79 80```bash81unlimited-ocr-rdna4 prepare82unlimited-ocr-rdna4 prepare --dry-run --json83unlimited-ocr-rdna4 prepare --model-dir /data/models/unlimited-ocr84```85 86Preparation is serialized and idempotent. It uses a unique partial directory, validates the exact manifest and file set,87and refuses unknown, extra, symlinked, or mismatched files instead of executing them.88 89### `doctor`90 91Reports the active PyTorch/HIP stack, visible devices, RDNA 4 architecture, BF16 capability, and model readiness.92 93```bash94unlimited-ocr-rdna4 doctor95unlimited-ocr-rdna4 doctor --device 0 --require-model96unlimited-ocr-rdna4 doctor --json97```98 99### `run`100 101Parses one image or PDF and publishes untrusted model output atomically. The content may include Markdown and raw HTML;102do not render it in a privileged origin without sanitization.103 104```bash105unlimited-ocr-rdna4 run --input scan.png106unlimited-ocr-rdna4 run --input scan.png --output scan.md --device 0107unlimited-ocr-rdna4 run --input report.pdf --start-page 21 --max-pages 10108```109 110Important flags:111 112| Flag | Default | Meaning |113|---|---:|---|114| `--device` | `$UNLIMITED_OCR_DEVICE` or `0` | ROCm ordinal or stable ROCr UUID |115| `--mode` | `gundam` | `gundam` for detailed single-page parsing; `base` for lower-detail input |116| `--max-length` | `4096` | Total input + output sequence limit; maximum `32768` |117| `--dpi` | `200` | PDF rendering resolution |118| `--max-pages` | `20` | Per-run PDF safety cap |119| `--max-page-pixels` | `60000000` | Per-page rendered-pixel cap |120| `--max-total-pixels` | `400000000` | Aggregate rendered-pixel cap |121| `--max-page-rendered-mib` | `512` | Per-page rendered temporary-byte cap |122| `--max-rendered-mib` | `2048` | Aggregate rendered temporary-byte cap |123| `--force` | off | Replace an existing output file |124| `--json` | off | Stable machine-readable summary on stdout |125| `--quiet` | off | Suppress progress diagnostics |126 127OCR content is written to the output file. Progress and warnings go to stderr. Human or JSON summaries go to stdout.128 129## GPU selection130 131PyTorch uses the CUDA-compatible API name on ROCm. `--device` sets `ROCR_VISIBLE_DEVICES` before PyTorch is imported, so132the process sees one logical `cuda:0` backed by the selected AMD card.133 134For ordinary one-GPU systems:135 136```bash137unlimited-ocr-rdna4 run --input page.png --device 0138```139 140For multi-GPU systems, a stable ROCr UUID avoids dependence on enumeration order:141 142```bash143unlimited-ocr-rdna4 run --input page.png --device GPU-0123456789abcdef144```145 146Run `unlimited-ocr-rdna4 doctor` before choosing a device. Do not assume another application's GPU numbering matches147ROCm's ordinal numbering.148 149## What the RDNA 4 adaptation changes150 151- Pins the Baidu checkpoint and verifies the complete expected local model tree before custom code loads.152- Uses AMD's production ROCm 7.2.1 PyTorch, torchvision, and Triton wheels for Python 3.12.153- Isolates one `gfx1200`/`gfx1201` GPU before importing PyTorch.154- Replaces unsafe `eval()` calls in optional model-output geometry parsing with `ast.literal_eval()`.155- Makes one internal mask transfer follow the active tensor device.156- Supplies the missing all-ones attention mask and pad token for single-sequence generation.157- Loads only the prepared local model during inference and enables Hugging Face offline mode.158- Avoids vLLM, SGLang, quantization, tensor parallelism, and unverified custom serving kernels.159- Removes only complete, exact layout sentinel pairs without rewriting recognized Unicode or TeX.160- Detects obvious terminal repetition and warns without silently rewriting recognition content.161 162The model weights and mathematical operators are unchanged. PyTorch's ROCm backend provides the RDNA 4 kernels.163 164## Configuration165 166Precedence is flags, then environment variables, then XDG defaults.167 168| Environment variable | Purpose |169|---|---|170| `UNLIMITED_OCR_DEVICE` | Default ROCm ordinal or UUID |171| `UNLIMITED_OCR_MODEL_DIR` | Prepared model directory |172| `XDG_DATA_HOME` | Default model storage root |173| `XDG_CACHE_HOME` | Wheel, Hugging Face, and temporary-work cache root |174| `NO_COLOR` | Accepted implicitly; the CLI currently emits no color |175 176This project contains no analytics or telemetry code.177 178## Safety and limitations179 180- The default 4096 sequence limit includes visual-prefill tokens. Dense pages may require a larger value.181- PDF mode uses permissively licensed PDFium bindings and renders one bounded page at a time. A failed run leaves no182  published partial document; retry with `--start-page` to resume manually.183- Generative OCR can omit or hallucinate content. Verify consequential documents against the source.184- Rotated text and repetitive pages are known upstream weak spots.185- Only the configuration above has completed the full repository acceptance test. Newer ROCm/PyTorch versions may work,186  but `doctor` reports them as unverified until measured.187- The hardware guard accepts `gfx1200`, but reports `hardware_verified=false`; only `gfx1201` has completed this GPU gate.188- The bootstrap currently supports Linux x86_64, Python 3.12, and host ROCm 7.2.1.189 190See [SECURITY.md](SECURITY.md) before processing untrusted documents.191 192## Development193 194CPU-only tests do not install PyTorch. The commands below are convenient for development; the release bootstrap is the195hash-locked installation path.196 197```bash198python3.12 -m venv .venv-dev199.venv-dev/bin/pip install -e '.[dev]'200.venv-dev/bin/ruff check .201.venv-dev/bin/ruff format --check .202.venv-dev/bin/pytest203.venv-dev/bin/python -m build204.venv-dev/bin/twine check dist/*205```206 207The GPU smoke gate is intentionally separate. It uses tracked, hash-checked fixtures; parses structured JSON; runs the208image twice in fresh processes; and checks image/PDF output hashes, structure, stack identity, revision, VRAM, and process209release. On the validated host, run it in a root-created private network namespace:210 211```bash212VALIDATION_NETWORK_ISOLATED=1 \213UNLIMITED_OCR_DEVICE=GPU-0123456789abcdef \214AMD_SMI_GPU=3 \215./scripts/validate-smoke.sh216```217 218## Credits and license219 220- Model and model code: [Baidu Unlimited-OCR](https://github.com/baidu/Unlimited-OCR), MIT licensed.221- ROCm evaluation and batching research: [AIwork4me/Unlimited-OCR-ROCm](https://github.com/AIwork4me/Unlimited-OCR-ROCm).222- AMD wheel source and compatibility guidance: [ROCm documentation](https://rocm.docs.amd.com/).223- PDF rendering: [pypdfium2](https://pypi.org/project/pypdfium2/) and PDFium, under permissive licenses.224 225The runtime is MIT licensed. Baidu's original copyright and license are preserved; see226[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).227