valira-ai/slo-ocr-benchmark

slOCR: Slovenian-Language OCR Benchmark 🏔️

A comprehensive leaderboard evaluating 20 OCR systems on 632 Slovene document pages across 6 categories: Academic, Handwriting, Historical, Billboard, Web, and Textbook.

Models evaluated: proprietary API systems (Claude, GPT, Gemini, Mistral), open-source VLMs (Qwen3, Kimi, InternVL, GaMS, ...), OCR-specialized models (olmOCR, Nanonets, GLM-OCR, Nemotron-Parse), and traditional pipelines (Docling, Marker, MinerU).

Metrics: CER, WER, BoW F1, Diacritic Micro/Macro F1, TEDS, Equation TER, Figure Recall, averaged per category into a composite Category Score and then across categories into an Overall score.

95% confidence intervals via 10,000-sample paired document bootstrap are available via the display mode toggle.