BDRC/PageToChar-Outline-benchmark

Page-overlap benchmark

This benchmark dataset for detecitng the text boundaries in page where text boundaires overlap while transferring BDRC page-level outlines into character-level outlines over OCR text.

Most outlined texts line up with page boundaries. For those cases the character span is the first character of the start page through the last character of the end page. Some texts do not: one work can end mid-page while another begins on the same page, or several short works can start and end on one page (one text ends, the next occupies only a few lines, then another begins).

To locate those mid-page boundaries we compare a rule-based finder, an LLM (Gemini), and a custom fine-tuned mmBERT boundary detector. This benchmark measures which method recovers the gold offsets more accurately.

Dataset

96 OCR pages from 77 works & 77 scan volumes, split evenly by overlap type:

• —48 single-overlap pages — two texts share the page (multiple: false, one break)
• —48 multiple-overlap pages — three or more texts share the page (multiple: true, two or more breaks)

Together the gold labels contain 149 text breaks. Pages come from real overlapping OCR, not synthetic splits.

page_benchmark/
  single/page1.txt
  single/page1.json
  …
  multiple/page1.txt
  multiple/page1.json
  manifest.json

pageN.json looks like:

{
  "breaks": [
    {
      "left_context": "དགེ་བར་གྱུར་ཅིག།",
      "text_break_string": " །། ༄། །",
      "contextual_break_string": "དགེ་བར་གྱུར་ཅིག། །། ༄། །",
      "char_offset": 291
    }
  ],
  "multiple": false
}

• —char_offset is the exclusive page-local end of the ending work's terminal marker
• —manifest.json has w_id, volume, i_id, image number, and outline segment ids

license: cc ---