vadik82/hayai-ocr-v25-onnx

Hayai OCR v2.5 Nova — ONNX

ONNX export of Hayai OCR v2.5 Nova (Apache-2.0): SigLIP2 NaFlex encoder + 12-layer causal decoder with the DSCProjector (pixel unshuffle 4→1 — the decoder sees 4× fewer vision tokens than v2.1). The upstream repository ships PyTorch weights only; this repo provides ready-to-run ONNX Runtime graphs.

Files

hayai_encoder_fp16.onnx   # SigLIP2 NaFlex: pixel_values/attention_mask/pos_embeds [B,256,768] → visual_features [B,256,768]
hayai_decoder_fp16.onnx   # decoder: visual_tokens [B,M',3072] + text_token_ids/cos/sin/mask → logits [B,N,16004]
tokenizer.json
position_base.npy         # 16×16×768 base grid for NaFlex position interpolation

• —M' = ceil(h/2)·ceil(w/2) — dynamic decoder vision length (the 4-to-1 pixel-unshuffled tokens; the permutation itself runs host-side).
• —The encoder interface is identical to v2.1; cos/sin are built on the compressed grid, the mask is block-causal over L = M' + N.
• —fp32 variants (256/512-patch encoders) and the export script are available in the mangosh repository (export_onnx25.py); parity with the PyTorch reference generate() is verified.

Performance

Measured on real manga crops (64 crops from a production pipeline, ONNX Runtime 1.30, CUDA, fp16 graphs, RTX 3060, sequential decode; max_new_tokens 48):

End-to-end, full manga volume (199 pages, production pipeline, same GPU):

The theoretical 4× token reduction does not translate into 4× wall time: per-step overheads (ORT dispatch, the 512×16004 vocabulary head) are fixed and the encoder is unchanged (2× cost at the 512-patch budget).

Text quality (upstream numbers, JMangaBench_Mixed crop benchmark, CER ↓):

At an equal 256-patch budget v2.5 loses some detail on dense vertical text; raising the budget to 512 is cheap for v2.5 (4× token compression) and restores/beats v2.1 accuracy. On typical dialogue crops the two models are on par (each misreads different glyphs). The upstream PyTorch baseline reports relative latency 1.00× / 1.19× / 1.34× for 256/384/512 patches (T4 GPU) — the ONNX graphs above keep the same scaling.

For a v2.1 KV-cache variant (decode O(1) per token), see the companion repo vadik82/hayai-ocr-v2-onnx — measured ×2.4–3.0 end-to-end on the OCR stage (int8, greedy).

License and attribution

Weights — JustANormalTinkerer/hayai-ocr-v2.5-nova, Apache-2.0. The ONNX graphs are derived from the same weights, same license.