Spacewanderer8263/Proxy3D-annotations

SpaceSpan Dataset

SpaceSpan is a large-scale dataset curated for aligning 3D proxy representations with Vision-Language Models (VLMs), introduced in the paper Proxy3D: Efficient 3D Representations for Vision-Language Models via Semantic Clustering and Alignment.

The dataset incorporates heterogeneous visual information into a unified format to support multi-stage training for developing spatial intelligence. It enables models to progress from simple image-text alignment to complex 3D reasoning tasks, such as 3D visual question answering (VQA) and visual grounding.

**Project Page** | **GitHub** | **Paper**

Dataset Description

The SpaceSpan dataset (specifically the SpaceSpan-318K version) supports four progressive training stages:

• —Stage 1: Initial spatial alignment.
• —Stage 2-3: Intermediate spatial reasoning development.
• —Stage 4: Full-scale 3D reasoning.

Directory Structure

The dataset can be organized as follows:

data/               # Training and inference data
├── icon_image_embeds_qwen25.pt
├── number_image_embeds_qwen25.pt
├── stage_1_train.json
├── stage_2_train.json
├── stage_3_train.json
├── stage_4_train_318K.json
├── pointmaps_wo_markers
├── poses
└── ... 

Citation

If you find this dataset useful for your research, please cite the following paper:

@article{proxy3d2026,
  title={Proxy3D: Efficient 3D Representations for Vision-Language Models via Semantic Clustering and Alignment},
  author={Jiang, Jerry and Sun, Haowen and Gudovskiy, Denis and Nakata, Yohei and Okuno, Tomoyuki and Keutzer, Kurt and Zheng Wenzhao},
  journal={arXiv preprint arXiv:2605.08064},
  year={2026}
}