stanford-star/the-join

The Join

A broad collection of relational databases spanning many domains (academic, e-commerce, finance, sports, biomedical, government, text2sql, and more), ported to the RelBench manifest format. The Join is built for pretraining relational/tabular foundation models: each database is self-describing and tasks ship labels as-is for large-scale pretraining rather than held-out benchmarking.

Each dataset lives in its own subdirectory in the self-describing manifest layout (plain parquet + manifest.yaml):

<dataset>/
  manifest.yaml                 # tables, primary keys, foreign-key graph
  schema.svg                    # ER diagram
  db/*.parquet                  # relational tables (plain parquet)
  tasks/<task>/manifest.yaml    # task spec (kind: forecast | autocomplete)
  tasks/<task>/train.parquet

`STATS/databases.parquet` catalogs every database: domain, description, table / row / task counts, size, license, and source. `STATS/tasks.parquet` does the same per task: kind, task type, entity and target columns, window, metric, and split sizes. Both are browsable in the dataset viewer above.

Load with relbench.load_dataset("relbench/join/<dataset>") / relbench.load_task("relbench/join/<dataset>", "<task>"), or read any subdir's parquet directly with pandas/duckdb.

Tasks are kind: forecast (labels shipped as-is) or kind: autocomplete. This is a train-only release: tasks ship a single train.parquet and val_timestamp / test_timestamp are null, designed for pretraining, not held-out evaluation.

Databases whose source data overlaps with RelBench and dbinfer benchmark datasets are excluded.

Licensing

The collection is released under CC BY-SA 4.0. Individual databases inherit the licenses of their original sources; see the license and source_url columns of `STATS/databases.parquet` for per-database attribution.