docling-project/PubTabNet_OTSL

Dataset Card for PubTabNet_OTSL

Dataset Description

• —Homepage: https://ds4sd.github.io
• —Paper: https://arxiv.org/pdf/2305.03393

Dataset Summary

This dataset is a conversion of the original PubTabNet into the OTSL format presented in our paper "Optimized Table Tokenization for Table Structure Recognition". The dataset includes the original annotations amongst new additions.

Dataset Structure

• —cells: origunal dataset cell groundtruth (content).
• —otsl: new reduced table structure token format
• —html: original dataset groundtruth HTML (structure).
• —html_restored: generated HTML from OTSL.
• —cols: grid column length.
• —rows: grid row length.
• —image: PIL image

OTSL Vocabulary:

OTSL: new reduced table structure token format More information on the OTSL table structure format and its concepts can be read from our paper. Format of this dataset extends work presented in a paper, and introduces slight modifications:

• —"fcel" - cell that has content in it
• —"ecel" - cell that is empty
• —"lcel" - left-looking cell (to handle horizontally merged cells)
• —"ucel" - up-looking cell (to handle vertically merged cells)
• —"xcel" - 2d span cells, in this dataset - covers entire area of a merged cell
• —"nl" - new line token

Data Splits

The dataset provides three splits

• —train
• —val

Additional Information

Dataset Curators

The dataset is converted by the Deep Search team at IBM Research. You can contact us at deepsearch-core@zurich.ibm.com.

Curators:

• —Maksym Lysak, @maxmnemonic
• —Ahmed Nassar, @nassarofficial
• —Christoph Auer, @cau-git
• —Nikos Livathinos, @nikos-livathinos
• —Peter Staar, @PeterStaar-IBM

Citation Information

@misc{lysak2023optimized,
      title={Optimized Table Tokenization for Table Structure Recognition}, 
      author={Maksym Lysak and Ahmed Nassar and Nikolaos Livathinos and Christoph Auer and Peter Staar},
      year={2023},
      eprint={2305.03393},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}```