CoolFace
Apppublic

maqiuping59/table_markdown

sourceHugging Faceupdated 1y agoView on Hugging Face
0likes
App README

Metric Card for Table Markdown Metrics

Metric Description

This metric evaluates the accuracy of table data extraction or generation by comparing predicted tables with reference tables. It calculates:

  1. 1.Precision: The ratio of correctly predicted cells to the total number of cells in the predicted table
  2. 2.Recall: The ratio of correctly predicted cells to the total number of cells in the reference table
  3. 3.F1 Score: The harmonic mean of precision and recall

How to Use

This metric requires predictions and references as inputs in Markdown table format.

python
>>> table_metric = evaluate.load("table_markdown")
>>> results = table_metric.compute(
...     predictions="|  | lobby | search | band | charge | chain ||--|--|--|--|--|--|| desire | 5 | 8 | 7 | 5 | 9 || wage | 1 | 5 | 3 | 8 | 5 |",
...     references="|  | lobby | search | band | charge | chain ||--|--|--|--|--|--|| desire | 1 | 6 | 7 | 5 | 9 || wage | 1 | 5 | 2 | 8 | 5 |"
... )
>>> print(results)
{'precision': 0.7, 'recall': 0.7, 'f1': 0.7, 'true_positives': 7, 'false_positives': 3, 'false_negatives': 3}

Inputs

  • —predictions (str): Predicted table in Markdown format.
  • —references (str): Reference table in Markdown format.

Output Values

  • —precision (float): Precision score. Range: [0,1]
  • —recall (float): Recall score. Range: [0,1]
  • —f1 (float): F1 score. Range: [0,1]
  • —true_positives (int): Number of correctly predicted cells
  • —false_positives (int): Number of incorrectly predicted cells
  • —false_negatives (int): Number of cells that were not predicted

Examples

Example - Complex table comparison:

python
>>> table_metric = evaluate.load("table_markdown")
>>> results = table_metric.compute(
...     predictions="""
... |  | lobby | search | band |
... |--|-------|--------|------|
... | desire | 5 | 8 | 7 |
... | wage | 1 | 5 | 3 |
... """,
...     references="""
... |  | lobby | search | band |
... |--|-------|--------|------|
... | desire | 5 | 8 | 7 |
... | wage | 1 | 5 | 3 |
... """
... )
>>> print(results)
{'precision': 1.0, 'recall': 1.0, 'f1': 1.0, 'true_positives': 6, 'false_positives': 0, 'false_negatives': 0}

Limitations and Bias

  1. 1.The metric assumes that tables are well-formed in Markdown format
  2. 2.The comparison is case-sensitive
  3. 3.The metric does not handle merged cells or complex table structures
  4. 4.The metric treats each cell as a separate unit and does not consider the semantic meaning of the content

Citation(s)

bibtex
@article{ChineseChartExtractor,
  title={Research on Chinese Chart Data Extraction Methods},
  author={Qiuping Ma,Hangshuo Bi,Qi Zhang,Xiaofan Zhao},
  journal={None},
  volume={0},
  pages={0--0},
  year={2025}
}

Further References