dougvk/chandra-ocr-2-BF16-GGUF-RDNA4

Chandra OCR 2 BF16 GGUF — one-GPU RDNA 4 validation

Unofficial, reproducible BF16 GGUF conversion of `datalab-to/chandra-ocr-2`, validated end to end with `llama.cpp` on one AMD Radeon RX 9070 XT (gfx1201).

This is a container conversion, not a fine-tune, new model, or claim of improved OCR quality. The main GGUF preserves BF16 precision while omitting Chandra's unused multi-token-prediction (MTP) draft head. The vision projector is BF16. The original model, architecture, training, and authorship belong to Datalab.

Files

The complete machine-readable provenance is in `manifest.json`.

Pinned provenance

• —Source model: datalab-to/chandra-ocr-2
• —Source revision: af93b47dba1b47b6640c86ccf487ed2260ab9a09
• —Source model.safetensors SHA-256: 0804568be9f099d6479fad9ed77a4da4611f3c1e7bc6e009af7dce45e8aa3847
• —Converter/runtime: ggml-org/llama.cpp
• —Converter revision: 8f5ab832ca7d8a7b4f23687693fb8b0ecbc227e7
• —chandra-ocr: 0.2.0

See `CONVERSION.md` for the exact commands and integrity checks.

Verified configuration

The GGUF format is not RDNA4-specific. gfx1201 is the hardware on which this exact pair completed image, PDF, financial-table, handwritten-document, deterministic-output, and lifecycle tests. Other llama.cpp-supported hardware may work but is not validated here.

Run with llama.cpp and the Chandra CLI

Build llama.cpp for your accelerator, then start a loopback server. These are the validated inference settings; replace the device selector as appropriate for your machine.

ROCR_VISIBLE_DEVICES=GPU-YOUR-STABLE-UUID HIP_VISIBLE_DEVICES=0 \
llama-server \
  --model chandra-ocr-2.BF16.gguf \
  --mmproj chandra-ocr-2.mmproj-bf16.gguf \
  --alias chandra \
  --host 127.0.0.1 \
  --port 18100 \
  --ctx-size 24576 \
  --n-gpu-layers 999 \
  --split-mode none \
  --main-gpu 0 \
  --flash-attn on \
  --fit off \
  --parallel 1 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --image-min-tokens 1024

In another shell:

python3 -m venv .venv
.venv/bin/pip install 'chandra-ocr==0.2.0'

VLLM_API_BASE=http://127.0.0.1:18100/v1 \
VLLM_API_KEY=EMPTY \
VLLM_MODEL_NAME=chandra \
.venv/bin/python -m chandra.scripts.cli input.pdf output \
  --method vllm \
  --batch-size 1 \
  --max-workers 1 \
  --max-retries 2 \
  --max-output-tokens 12384 \
  --no-images \
  --no-html

Do not expose an unauthenticated llama.cpp server to a public network. Generative OCR can omit or hallucinate content; verify consequential documents against their source.

Validation result and limits

The exact-pinned conversion matched the public comparison BF16 tensor schema. All projector tensor payloads matched. In the main file, every tensor payload matched except twelve scalar F32 values across ten ssm_a tensors; the maximum absolute difference was 9.5367431640625e-07. Accepted OCR outputs matched. There is no evidence that this conversion improves OCR quality over other correct BF16 conversions.

The useful contribution is the pinned, MTP-free artifact pair; complete provenance; one-16-GB-GPU validation; and a reproducible launch envelope.

License and attribution

Chandra OCR 2 weights use Datalab's AI Pubs Open RAIL-M License (Modified). It contains usage, redistribution, commercial, competitive-use, attribution, and share-alike conditions. Read `LICENSE` in full before using or redistributing these files. The license and its restrictions apply to this derivative conversion.

Modified-file notice: the original Chandra OCR 2 checkpoint was converted to GGUF at the pinned llama.cpp revision; the unused MTP draft head was omitted from the main GGUF; model tensor precision otherwise remains BF16. No Datalab endorsement is claimed.