Alvin16/SpaceSense-Bench

SpaceSense-Bench: Multi-Modal Spacecraft Perception and Pose Estimation Dataset

**Project Page** | **Paper** | **Toolkit & Code**

SpaceSense-Bench is a high-fidelity simulation-based multi-modal (RGB, Depth, LiDAR Point Cloud) dataset for spacecraft component-level semantic understanding, containing 136 satellite models with synchronized multi-modal data.

Update (2026-05-19). Following issue #5, the pose_ground_truth.csv for all 136 spacecraft has been regenerated to fix a frame-timing misalignment with the rendered sensor data. The RGB / depth / segmentation / lidar files themselves are unchanged. All raw/<sat>.tar.gz archives have been re-uploaded with the corrected pose CSVs. Thanks to @zstar239 for the original report.

Dataset Overview

Sample Usage

The SpaceSense-Toolkit provides tools for converting raw data to standard formats and visualizing the results.

Installation

pip install -r requirements.txt

Conversion and Visualization

# Visualize the raw data
python SpaceSense-Toolkit/visualize/raw_data_web_visualizer.py --raw-data data_example

# Convert to Semantic-KITTI (3D segmentation)
python SpaceSense-Toolkit/convert/airsim_to_semantickitti.py --raw-data data_example --output output/semantickitti --satellite-json SpaceSense-Toolkit/configs/satellite_descriptions.json

# Convert to MMSegmentation (2D segmentation)
python SpaceSense-Toolkit/convert/airsim_to_mmseg.py --raw-data data_example --output output/mmseg

# Convert to YOLO (Object detection)
python SpaceSense-Toolkit/convert/airsim_to_yolo.py --raw-data data_example --output output/yolo

Data Modalities

Coordinate System & Units

Sensor Configuration

Camera (cam0)

• —Resolution: 1024 x 1024
• —FOV: 50 degrees
• —Image types captured: RGB (Scene, type 0), Segmentation (type 5), Depth (DepthPerspective, type 2)
• —TargetGamma: 1.0
• —Mounting offset: 1 m in front of the service spacecraft body origin

LiDAR

• —Range: 300 m
• —Channels: 256
• —Vertical FOV: -20 to +20 degrees
• —Horizontal FOV: -20 to +20 degrees
• —Data frame: SensorLocalFrame

Data Split (Zero-shot / OOD)

The training and validation sets contain completely non-overlapping satellite models, so validation performance reflects zero-shot generalization to unseen spacecraft.

Data Organization

Each .tar.gz file in the raw/ folder contains data for one satellite. The toolkit expects trajectory folders containing image/, seg/, depth/, lidar/, and pose_ground_truth.csv.

Typical layout:

<satellite_name>/ or <timestamp>_<satellite_name>/
├── approach_front/
│   ├── image/              # RGB images (.png)
│   ├── depth/              # Depth maps (.npz, int32, mm, key = depth)
│   ├── seg/                # Semantic masks (.png, RGB color encoding)
│   ├── lidar/              # Point clouds (.asc)
│   └── pose_ground_truth.csv
├── approach_back/
├── orbit_xy/
└── ...

Some HuggingFace exports may contain one additional nesting level:

<satellite_name>/
└── <satellite_name>/
    ├── approach_front/
    ├── approach_back/
    ├── orbit_xy/
    └── ...

Semantic Class Definitions

License

This dataset is released under the CC-BY-NC-4.0 license. Non-commercial use only.

Citation

@article{SpaceSense-Bench,
	title={SpaceSense-Bench: A Large-Scale Multi-Modal Benchmark for Spacecraft Perception and Pose Estimation},
	author={Aodi Wu, Jianhong Zuo, Zeyuan Zhao, Xubo Luo, Ruisuo Wang, Xue Wan},
	year={2026},
	url={https://arxiv.org/abs/2603.09320}
}