McH04/Rotowire-Text-to-Table

RotoWire Corrected Test Set

Dataset Description

This is the corrected test set for the RotoWire dataset, released as part of the Map&Make: Schema Guided Text to Table Generation paper (ACL 2025). The original RotoWire dataset contained hallucination errors where the ground truth tables included incorrect or fabricated statistics. This corrected version provides a cleaner benchmark for text-to-table generation tasks.

Motivation: Why We Need This Corrected Dataset

Data Quality Issues in Existing Datasets

The RotoWire dataset has been widely used for structured data generation tasks, but significant data quality issues have emerged as the dataset was repurposed for different tasks:

1. Original RotoWire Dataset (Wiseman et al., 2017): The original RotoWire dataset was created for table-to-text generation - generating game summaries from structured statistics tables. The ground truth tables in this dataset contained basketball game statistics for teams and players.

2. Repurposing for Text-to-Table (Wu et al., 2022): Wu et al. (2022) pioneered the reverse task by repurposing the RotoWire dataset for text-to-table generation - extracting structured tables from game summaries. However, this repurposed dataset contained substantial errors in the ground truth tables, including:

• —Hallucinated statistics not supported by the text
• —Incorrect player or team information
• —Fabricated numerical values
• —Missing information that should have been present

3. Issues in Strucbench's Correction Attempt (Tang et al., 2024): Strucbench (2024) built upon Wu et al.'s work and attempted to correct these issues. However, our comprehensive analysis revealed that the corrections were incomplete and, in some cases, introduced new errors rather than fixing existing ones.

Quantitative Analysis of Errors

Our comprehensive error analysis (shown in Table 1 of our paper) reveals the extent of contamination:

Comparing Original to Strucbench:

• —Team tables: 1,219 hallucinated cells + 1,271 missing information cells
• —Player tables: 1,390 hallucinated cells + 1,270 missing information cells

Comparing Original to Our Corrected Version:

• —Team tables: 613 hallucinated cells + 1,137 missing information cells were fixed
• —Player tables: 7,310 hallucinated cells + 1,752 missing information cells were fixed

Comparing Strucbench to Our Corrected Version:

• —Team tables: 721 hallucinated cells + 1,247 missing information cells remained/were introduced
• —Player tables: 8,104 hallucinated cells + 2,666 missing information cells remained/were introduced

These numbers demonstrate that even the Strucbench correction attempt left substantial errors and in some cases introduced new ones. Our corrected version addresses these issues through careful manual verification and correction.

Impact on Research

Using contaminated ground truth for evaluation leads to:

• —Unreliable model performance metrics
• —Unfair comparison between different approaches
• —Models potentially learning from incorrect examples
• —Misleading conclusions about model capabilities

This corrected dataset provides researchers with a clean benchmark for evaluating text-to-table generation models.

Dataset Structure

Data Instances

Each instance in the dataset contains:

• —id: Unique identifier for the sample (0-727)
• —text: Natural language game summary/description
• —player_table: Player statistics in structured format
• —team_table: Team statistics in structured format

Data Fields

Main Fields

• —id (int): Sample index
• —text (string): Game summary text
• —player_table (dict): Player statistics table with:
• —headers (list): Column names (e.g., ["Player", "Points", "Assists", "Rebounds", ...])
• —rows (list of lists): Player statistics data
• —team_table (dict): Team statistics table with:
• —headers (list): Column names (e.g., ["Team", "Wins", "Losses", "Total points"])
• —rows (list of lists): Team statistics data

Player Table Columns (may vary by sample)

• —Player: Player name
• —Points: Points scored
• —Assists: Assists made
• —Total rebounds: Total rebounds
• —Steals: Steals made
• —Blocks: Blocked shots
• —Field goals made/attempted
• —3-pointers made/attempted
• —Free throws made/attempted
• —Minutes played

Team Table Columns

• —Team: Team name
• —Wins: Season wins
• —Losses: Season losses
• —Total points: Points scored in the game

Example

{
  "id": 0,
  "text": "The Atlanta Hawks (46 - 12) beat the Orlando Magic (19 - 41) 95 - 88 on Friday...",
  "player_table": {
    "headers": ["Player", "Assists", "Points", "Total rebounds", "Steals"],
    "rows": [
      ["Nikola Vucevic", "", "21", "15", ""],
      ["Al Horford", "4", "17", "13", "2"],
      ["Jeff Teague", "7", "17", "", "2"]
    ]
  },
  "team_table": {
    "headers": ["Team", "Losses", "Total points", "Wins"],
    "rows": [
      ["Hawks", "12", "95", "46"],
      ["Magic", "41", "88", "19"]
    ]
  }
}

Dataset Statistics

• —Total Samples: 728
• —Source: RotoWire (basketball game summaries)
• —Split: Test set

Usage

Loading the Dataset

from datasets import load_dataset

# Load the dataset
dataset = load_dataset("YOUR_USERNAME/YOUR_DATASET_NAME")

# Access a sample
sample = dataset['train'][0]
print(f"Text: {sample['text']}")
print(f"Player table headers: {sample['player_table']['headers']}")
print(f"Player table rows: {sample['player_table']['rows']}")

Working with Tables

# Convert player table to pandas DataFrame
import pandas as pd

def table_to_dataframe(table):
    """Convert table structure to pandas DataFrame"""
    return pd.DataFrame(table['rows'], columns=table['headers'])

# Use it
sample = dataset['train'][0]
player_df = table_to_dataframe(sample['player_table'])
team_df = table_to_dataframe(sample['team_table'])

print(player_df)
print(team_df)

Iterating Through Samples

# View all samples
for i, sample in enumerate(dataset['train']):
    print(f"\n{'='*80}")
    print(f"Sample {i}")
    print(f"{'='*80}")
    print(f"Text: {sample['text'][:200]}...")
    print(f"\nPlayer Table Shape: {len(sample['player_table']['rows'])} rows x {len(sample['player_table']['headers'])} columns")
    print(f"Team Table Shape: {len(sample['team_table']['rows'])} rows x {len(sample['team_table']['headers'])} columns")

Use Cases

This dataset is designed for:

• —Text-to-Table Generation: Extract structured tables from natural language game summaries
• —Information Extraction: Identify and extract player and team statistics from text
• —Benchmark Evaluation: Clean test set for evaluating text-to-table models without contaminated ground truth

Related Datasets

In addition to this corrected RotoWire test set, our Map&Make paper also evaluates on:

• —Livesum Dataset: A dataset requiring numerical aggregation for text-to-table generation. Available at: https://github.com/HKUST-KnowComp/LiveSum/tree/main

Citation

If you use this corrected dataset, please cite:

@inproceedings{ahuja-etal-2025-map,
    title = "Map{\&}Make: Schema Guided Text to Table Generation",
    author = "Ahuja, Naman and Bardoliya, Fenil and Baral, Chitta and Gupta, Vivek",
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.acl-long.1460/",
    doi = "10.18653/v1/2025.acl-long.1460",
    pages = "30249--30262"
}

Related work on RotoWire dataset:

Original RotoWire dataset (table-to-text):

@inproceedings{wiseman2017challenges,
  title={Challenges in Data-to-Document Generation},
  author={Wiseman, Sam and Shieber, Stuart M and Rush, Alexander M},
  booktitle={Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing},
  pages={2253--2263},
  year={2017}
}

Text-to-table repurposing of RotoWire:

@inproceedings{wu-etal-2022-text,
    title = "Text-to-Table: A New Way of Information Extraction",
    author = "Wu, Xueqing and Zhang, Jiacheng and Li, Hang",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    year = "2022",
    pages = "2518--2533",
    url = "https://aclanthology.org/2022.acl-long.180/"
}

Strucbench correction attempt:

@inproceedings{tang-etal-2024-struc,
    title = "Struc-Bench: Are Large Language Models Good at Generating Complex Structured Tabular Data?",
    author = "Tang, Xiangru and Zong, Yiming and Phang, Jason and Zhao, Yilun and Zhou, Wangchunshu and Cohan, Arman and Gerstein, Mark",
    booktitle = "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)",
    year = "2024",
    pages = "12--34",
    url = "https://aclanthology.org/2024.naacl-short.2/"
}

License

MIT License

Notes

• —Empty strings in table cells indicate missing or not applicable data
• —Player and team statistics columns may vary across samples
• —This corrected test set contains 728 samples
• —For more details on the corrections and methodology, see the Map&Make paper
• —Code and additional resources: https://coral-lab-asu.github.io/map-make