kaan84/sinhala-print-trocr

sinhala-print-trocr

TrOCR fine-tune for printed Sinhala text lines, with a repaired decoder tokenizer that preserves Zero Width Joiner.

Use this repo. The predecessor, `kaan84/sinhala-print-trocr-pilot`, is frozen and contains a tokenizer defect; it is kept only as the published record of that finding.

Scope

Input is a single line image (~2600×128). This is not a page or document recogniser — see Real documents for what page-level use requires.

What changed

The predecessor's BertNormalizer ran with clean_text=True, deleting all Unicode Cf characters including U+200D ZWJ — the character that forms Sinhala conjuncts such as ශ්‍රී. Two changes were needed, and the order matters:

1. 1.Vocabulary first. WordPiece maps an entire word to [UNK] if any character is missing from the vocabulary. Patching the normalizer alone would have made output strictly worse. Five tokens were injected directly into model.vocab (32,000 → 32,005). add_tokens() does not work here: it creates standalone AddedTokens that bypass WordPiece, so the decoder inserts spaces around them.
2. 2.Then `clean_text=False`.

New embedding rows were warm-started (##x copies the row for x; ZWJ takes the matrix mean), then fine-tuned.

tok = AutoTokenizer.from_pretrained("kaan84/sinhala-print-trocr")
s = "ශ්\u200Dරී"
tok.decode(tok.encode(s, add_special_tokens=False)) == s   # True

Results

Controlled ablation: identical base checkpoint, data, seed (42), epochs and effective batch size. The tokenizer is the only difference. Held-out 5% split, 349 lines.

Compare against control, not against the pilot — the pilot-to-fixed gap includes extra training and would overstate the change.

Reading these numbers honestly

• —The capability was restored and is used. The control arm cannot emit ZWJ at all; this model does, on 2 of 20 conjunct-bearing eval lines. That asymmetry is structural, not statistical.
• —Recognition did not improve. 0.1709 → 0.1690 char-CER is noise.
• —Exact conjunct recall is still 0. Two reasons, both measured: ZWJ is 0.118% of training graphemes, giving almost no signal for a freshly-initialised embedding; and at ~17% CER the surrounding words are usually wrong anyway, so an exact grapheme match rarely has the chance to occur.

The fix removes a hard ceiling. It does not improve accuracy at current error rates.

Metric note

g-CER uses grapheme clusters from `grapheme-kit`. Report it alongside codepoint CER, never instead of it: grapheme-CER runs higher when errors corrupt a cluster from inside (a stripped ZWJ) and lower when a whole multi-codepoint grapheme is misread. A number that falls when you switch units is a change of ruler, not a better model.

Limitations

• —Synthetic training and evaluation. Same generator, same five fonts. These numbers largely measure memorisation of one renderer.
• —Line-level only. Pages must be segmented first.
• —Conjunct placement is unsolved. The model can emit ZWJ; it rarely places it correctly.
• —Training data contains stray joiners. Two of the five injected tokens (' ', '##‼') arose from ZWJ adjacent to spaces and punctuation in the corpus. They are unreachable in practice (BertPreTokenizer splits on whitespace) but indicate upstream noise.
• —One source row had null text and was filtered.

Real documents

avishadilhara/sinhala-ocr-lk-acts-1010 contains full page scans (~1650×2450 to 2480×3509, 1,200–2,500 characters per page), while this model reads single lines. Pages must be segmented before recognition.

Preliminary, one page from the test split with Tesseract layout analysis: 37 lines detected, char-CER 0.4657. Page-level ground truth means concatenated line predictions conflate recognition errors with reading-order errors. A full 202-page benchmark is pending.

Next steps

1. 1.Reduce baseline CER — at ~17% on single-font synthetic data, it caps everything downstream.
2. 2.Test the sparsity hypothesis by oversampling the 315 conjunct-bearing lines.
3. 3.Full 202-page real-document benchmark.

Related work

• —Velayuthan & Sarveswaran (2025), Egalitarian Language Representation in Language Models: It All Begins with Tokenizers, COLING 2025.
• —Nisfer et al. (2026), grapheme-kit: Grapheme-Level Metrics and Text Processing for Multilingual NLP.

The defect fixed here is in the normalizer, not the pre-tokenizer. BertPreTokenizer preserves ZWJ and yields one pre-token per conjunct, so the pre-tokenization compression ceiling described in the COLING paper does not apply to this tokenizer.