logo-lab/trl-dlte

TRL-DLTE

Paper: arXiv:2606.09323 — TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders · Code: LOGO-CUHKSZ/TRL-Bench

Compositional Data-Lake Table Enrichment suite of TRL-Bench. A 47,772-table data lake derived from 1,379 TabFact and WikiTableQuestions parent tables, fragmented at four cumulative noise tiers (clean / schema / cell / hard). Each parent yields a seed query, a union target (additional rows), and a join target (additional columns); the system must recover both targets from the lake by retrieval, column alignment, and row matching.

This repository ships two configs:

• —`manifests`: lightweight metadata records (109,336 rows, ~900 KB) — every entity in the benchmark (parents, fragments, lake tables, query tasks, distractor IDs, splits, fragmentation config) under a unified (record_type, record_json) schema.
• —`lake`: the full 54,667-table content (1,379 parents + 16,548 fragments [5,516 queryseed + 5,516 targetunion + 5,516 targetjoin] + 36,740 distractors), each with `csvtext + structural metadata + provenance keys. The retriever's actual search space is the **47,772-table** subset (targetunion + targetjoin + distractor`); parents and query seeds are released alongside as pipeline I/O.

manifests schema

{
  "record_type": string,    # "parent" | "fragment" | "lake_table" | "split_assignment" |
                            # "query_task" | "ckan_distractor" | "fragmentation_config" |
                            # "gt_validation"
  "record_json": string,    # JSON-encoded record content
}

Counts:

lake schema

{
  "table_id": string,
  "kind": string,           # "parent_tabfact" | "parent_wtq" | "target_union" |
                            # "target_join" | "query_seed" | "distractor"
  "parent_id": string,      # empty for distractors
  "parent_source": string,  # "tabfact" | "wtq" | ""
  "noise_tier": int32,      # 0/1/2/3 for fragments; -1 for parents/distractors
  "fragment_type": string,  # "seed" | "union" | "join" | ""
  "split": string,          # "train" | "dev" | "test" | "" (distractors are unsplit)
  "csv_text": string,       # raw CSV content
  "n_rows": int32,
  "n_cols": int32,
}

table_maps schema

Per-fragment column-mapping records (16,548 rows; one per fragment table_id). Encodes the row-index and column-index back-pointers from each fragment to its parent table. Required for Stage-2 (alignment) and Stage-3 (merge) evaluation; the retrieval-only Stage-1 does NOT require these.

{
  "table_id": string,                # fragment id (matches `lake` table_id)
  "row_parent_idx_json": string,     # JSON-encoded list[int]: per-row parent-row idx
  "col_parent_idx_json": string,     # JSON-encoded list[int]: per-col parent-col idx
}

Original on-disk form is an .npz per table with two int32 arrays (row_parent_idx, col_parent_idx). The stager (trl_bench.data.stage._stage_dlte_task) reconstructs the .npz files from this config.

Quickstart

from datasets import load_dataset
import json

# Manifests (structural metadata)
m = load_dataset("logo-lab/trl-dlte", "manifests")
print(m["train"]["record_type"][:5])
parents = m["train"].filter(lambda x: x["record_type"] == "parent")
print(json.loads(parents[0]["record_json"]))   # full parent record dict

# Lake content
lake = load_dataset("logo-lab/trl-dlte", "lake")
parents = lake["train"].filter(lambda x: x["kind"].startswith("parent_"))
distractors = lake["train"].filter(lambda x: x["kind"] == "distractor")
union_targets = lake["train"].filter(lambda x: x["kind"] == "target_union")

License

Umbrella CC-BY-SA-4.0 (driven by 390 WTQ-derived parents). Per-component licenses in LICENSES.md. Manifest records and the fragmentation config are TRL-Bench-derived under CC-BY-4.0; the lake content inherits per-source upstream licenses (TabFact: CC-BY 4.0; WTQ: CC-BY-SA 4.0; CKAN distractors: CC-BY 4.0 record / per-portal upstream).

Citation

If you use this dataset, please cite the TRL-Bench paper:

@article{pang2026trl,
  title={TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders},
  author={Pang, Wei and Jian, Xiangru and Li, Hehan and Yu, Zhixuan and Xue, Alex and Li, Jinyang and Dong, Zhengyuan and Zhao, Xinjian and Xu, Hao and Zhang, Chao and Cheng, Reynold and {\"O}zsu, M. Tamer and Yu, Tianshu},
  journal={arXiv preprint arXiv:2606.09323},
  year={2026}
}