wx91726/SpatialGen-Bench

1---2pretty_name: SpatialGen-Bench3language:4- en5size_categories:6- n<1K7task_categories:8- visual-question-answering9- image-to-image10license: other11tags:12- spatial-intelligence13- spatial-reasoning14- image-generation15- vision-language-models16- benchmark17dataset_info:18- config_name: default19  features:20  - name: id21    dtype: string22  - name: capability23    dtype: string24  - name: task25    dtype: string26  - name: image_path27    dtype: string28  - name: media_paths29    sequence: string30  - name: mask_path31    dtype: string32  - name: question33    dtype: string34  - name: answer35    dtype: string36  - name: answer_type37    dtype: string38  - name: choices39    sequence: string40  - name: source41    dtype: string42  - name: source_id43    dtype: string44  - name: metadata_json45    dtype: string46  splits:47  - name: test48    num_examples: 47049configs:50- config_name: default51  data_files:52  - split: test53    path: data/spatialgen_bench.jsonl54---55 56<p align="center">57  <video controls playsinline preload="metadata" poster="https://huggingface.co/datasets/wx91726/SpatialGen-Bench/resolve/main/assets/provise-overview-poster.webp" width="90%">58    <source src="https://huggingface.co/datasets/wx91726/SpatialGen-Bench/resolve/main/assets/provise-overview.mp4" type="video/mp4">59  </video>60</p>61 62<p align="center">63  <a href="https://arxiv.org/abs/2607.21072"><img src="https://img.shields.io/badge/arXiv-2607.21072-B31B1B?style=flat-square" alt="Paper"></a>64  <a href="https://zju-omniai.github.io/ProVisE/"><img src="https://img.shields.io/badge/Project-Page-1769AA?style=flat-square" alt="Project"></a>65  <a href="https://github.com/ZJU-OmniAI/ProVisE"><img src="https://img.shields.io/badge/Code-ProVisE-181717?style=flat-square&amp;logo=github&amp;logoColor=white" alt="Code"></a>66</p>67 68## Benchmark69 70<p align="center">71  <img src="assets/benchmark_overview.webp" alt="SpatialGen-Bench task overview across perception, understanding, reasoning, and interaction." width="100%">72</p>73 74Each record retains its source-task metric target as `integer`, `text`, `point`, `mask`, or `polyline`. The frozen visual-answer contract is available at [`protocols/spatialgen_bench.yaml`](protocols/spatialgen_bench.yaml), with runtime parsers in [ProVisE](https://github.com/ZJU-OmniAI/ProVisE).75 76## Quick Start77 78```python79from datasets import load_dataset80 81dataset = load_dataset("wx91726/SpatialGen-Bench", split="test")82print(dataset[0])83```84 85Download the complete media and evaluation package for local evaluation:86 87```bash88hf download wx91726/SpatialGen-Bench \89  --repo-type dataset \90  --local-dir SpatialGen-Bench91```92 93<details>94<summary><strong>Dataset structure and JSONL schema</strong></summary>95 96```text97SpatialGen-Bench/98|-- data/spatialgen_bench.jsonl99|-- benchmarks/{perception,understanding,reasoning,interaction}/100|-- protocols/spatialgen_bench.yaml101|-- SOURCES.md102`-- LICENSE103```104 105- `id`: stable sample identifier106- `capability`: one of the four capability levels107- `task`: subtask name108- `image_path`: primary input path relative to `benchmarks/`109- `media_paths`: ordered inputs for single- or multi-image tasks110- `mask_path`: optional ground-truth mask path111- `question`, normalized `answer`, `answer_type`, and `choices`112- `source` and `source_id`: upstream provenance113- `metadata_json`: serialized task-specific evaluator metadata114 115</details>116 117## Sources and License118 119SpatialGen-Bench annotations and protocol configuration are released under Apache-2.0. Source-derived media remains subject to the licenses and terms of its upstream datasets. [`SOURCES.md`](SOURCES.md) records task-level provenance, source links, and usage terms.120 121<details>122<summary><strong>Acknowledgements and upstream resources</strong></summary>123 124We sincerely appreciate [CountBench](https://teaching-clip-to-count.github.io/), [BLINK](https://huggingface.co/datasets/BLINK-Benchmark/BLINK), [EgoOrientBench](https://huggingface.co/datasets/jhCOR/EgoOrientBench), [VSR](https://github.com/cambridgeltl/visual-spatial-reasoning), [ViewSpatial-Bench](https://huggingface.co/datasets/lidingm/ViewSpatial-Bench), [MindCube](https://huggingface.co/datasets/MLL-Lab/MindCube), [VisWorld-Eval](https://github.com/thuml/Reasoning-Visual-World), [RoboAfford-Eval](https://huggingface.co/datasets/tyb197/RoboAfford-Eval), [ShareRobot-Bench](https://huggingface.co/datasets/BAAI/ShareRobot-Bench), [PhysBench](https://huggingface.co/datasets/USC-PSI-Lab/PhysBench), [SPHERE-VLM](https://sphere-vlm.github.io/), and [RefCOCOg](https://github.com/lichengunc/refer) for their public datasets, task designs, and evaluation resources.125 126</details>127 128## Citation129 130```bibtex131@article{wang2026showdonttell,132  title  = {Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text},133  author = {Wang, Xu and Yao, Kaixiang and Pan, Miao and Zhou, Xiaohe and Liu, Xuanyu and Zhang, Wenqi and Zhang, Xuhong},134  journal = {arXiv preprint arXiv:2607.21072},135  year   = {2026},136  url    = {https://arxiv.org/abs/2607.21072}137}138```139