yanmorona/UE4-Stereo

UE4-Stereo

Dataset Description

UE4-Stereo is a synthetic stereo vision dataset generated in Unreal Engine 4.27 using the Microsoft AirSim simulation framework. The dataset provides synchronized stereo RGB images, dense ground-truth depth maps, and full 6-DoF ground-truth poses. It is designed to support research and benchmarking in stereo matching, visual odometry, and visual SLAM, particularly in controlled indoor environments.

The dataset focuses on realistic indoor scenes with structural elements such as walls, pipes, pillars, industrial components, and furniture, while allowing precise control over camera configuration and motion dynamics.

The dataset contains multiple trajectory configurations, stereo baselines, and camera geometries, enabling evaluation under different acquisition conditions while preserving fully synchronized ground-truth information.

Dataset Overview

• —Number of scenes: 4 indoor scenes
• —Number of sequences: 32 acquisition sequences (8 per scene)
• —Total stereo image pairs: 36,838
• —Total RGB images: 73,676
• —Total size: approximately 81.8 GB
• —Trajectory type: Closed-loop trajectories
• —Sampling frequency: approximately 3 Hz (Scenes 1 and 2) and 10 Hz (Scenes 3 and 4)
• —Motion profile: recoverable from the timestamped ground-truth trajectories

Each sequence corresponds to a complete loop, where the drone takes off, follows a predefined trajectory, and lands at the same initial position.

Acquisition Setup

• —Simulation engine: Unreal Engine 4.27
• —Simulation framework: Microsoft AirSim
• —Platform: Virtual drone
• —Cameras: Stereo RGB cameras
• —Image resolution: 640 × 480 pixels
• —Camera intrinsics: Fixed across all sequences (fx = fy = 320.0 px, cx = 320.0 px, cy = 240.0 px, no lens distortion)
• —Stereo baseline: 24 cm and 32 cm (depending on the sequence)
• —Camera configuration: Parallel and convergent (5° inward each) setups

Drone motion and data acquisition were controlled via Python scripts using the AirSim API, available in the scripts/ folder. All data streams are temporally synchronized and referenced to the left camera.

Scenes and Sequences

Scene 1

Scene 1 represents an indoor industrial-like environment. It contains eight sequences with controlled variations in:

• —Loop direction (clockwise and counter-clockwise)
• —Lateral drone displacement (~1 m)
• —Stereo baseline (24 cm and 32 cm)
• —Camera convergence (parallel and 5° inward each)

The camera orientation remains fixed across all sequences in this scene.

Scene 2

Scene 2 represents a second indoor environment with similar structural complexity. It also contains eight sequences with variations in:

• —Loop direction (clockwise and counter-clockwise)
• —Lateral drone displacement (~1 m)
• —Stereo baseline (24 cm and 32 cm)
• —Camera convergence (parallel and 5° inward each)

Scene 2 introduces additional viewpoint and structural variations relative to Scene 1.

Scene 3

Scene 3 follows the same route configuration adopted in Scene 1, while using a continuous-velocity acquisition procedure that produces smoother trajectory transitions, together with denser temporal sampling and slight environmental variations.

The same trajectory variations are preserved:

• —Loop direction (clockwise and counter-clockwise)
• —Lateral drone displacement (~1 m)
• —Stereo baseline (24 cm and 32 cm)
• —Camera convergence (parallel and 5° inward each)

Scene 4

Scene 4 follows the same route configuration adopted in Scene 2, while using a continuous-velocity acquisition procedure that produces smoother trajectory transitions, together with denser temporal sampling and slight environmental variations.

The same trajectory variations are preserved:

• —Loop direction (clockwise and counter-clockwise)
• —Lateral drone displacement (~1 m)
• —Stereo baseline (24 cm and 32 cm)
• —Camera convergence (parallel and 5° inward each)

Sequence Statistics

Data Organization

The dataset is organized as follows:

scene_1/
  readme_scene1.txt
  sequence01/
    left_rgb/
    right_rgb/
    depth_npy/
    esquerda_depth_visual/
    associate_rgb.txt
    associate_rgbd.txt
    groundtruth.txt
  sequence02/
    ...
  ...
  sequence08/

scene_2/
  ...

scene_3/
  readme_scene3.txt
  sequence01/
    left_rgb/
    right_rgb/
    depth_npy/
    associate_rgb.txt
    associate_rgbd.txt
    groundtruth.txt
  sequence02/
    ...
  ...
  sequence08/

scene_4/
  ...

scripts/
README.md

Data Formats

RGB Images

• —Format: PNG
• —Resolution: 640 × 480
• —Channels: RGB (8-bit)
• —Stored separately for left and right cameras

Depth Maps

• —Metric depth: NumPy arrays (.npy)
• —Data type: float32
• —Units: meters
• —Resolution: 640 × 480
• —Planar depth, measured along the optical axis
• —Pixel-wise aligned with the left RGB image
• —A value of 0.0 marks an invalid pixel

• —Depth visualization: PNG images (esquerda_depth_visual/)
• —8-bit grayscale
• —Normalized for visualization only
• —Not suitable for quantitative evaluation
• —Provided for Scenes 1 and 2 only; the metric arrays are available for all sequences of all four scenes

Camera Poses

• —Format: Text (groundtruth.txt)
• —Pose representation:

timestamp tx ty tz qx qy qz qw

• —Position: (tx, ty, tz) in meters
• —Orientation: quaternion (qx, qy, qz, qw)
• —Coordinate system: AirSim world frame — right-handed North-East-Down, in meters, with the origin at the take-off point of each sequence. This differs from the native Unreal Engine world frame, which is left-handed and expressed in centimeters.
• —The recorded pose describes the vehicle body frame. To obtain the left camera optical frame, compose it with the rig extrinsics: a translation of (1.00, -B/2, 0.10) m in the body frame followed by a -23° pitch rotation, where B is the stereo baseline. Use +B/2 for the right camera.

The pose format follows the standard convention used by the TUM RGB-D dataset.

Association Files

Each sequence provides:

• —associate_rgb.txt: Synchronization between left and right RGB images
• —associate_rgbd.txt: Synchronization between left RGB images and depth maps

These files guarantee one-to-one correspondence between all data modalities.

Scripts

The scripts/ folder contains the Python scripts used to generate the dataset, the AirSim configuration file, and an example loader showing how to read images, depth maps, poses and camera parameters. See scripts/README.md for details.

Intended Use

This dataset is intended for:

• —Stereo disparity estimation
• —Visual odometry
• —Visual SLAM
• —Domain adaptation and synthetic-to-real transfer
• —Evaluation of robustness to camera geometry variations

Note: This is not a classification dataset. Folder names and file organization should not be interpreted as class labels.

License

This dataset is released under the Creative Commons Attribution 4.0 (CC BY 4.0) license.

Citation

If you use this dataset, please cite the corresponding Data in Brief article (under review). The citation will be updated upon acceptance. To cite the dataset itself:

@misc{morona2026ue4stereo,
  author    = {Morona, Yan Medeiros and Pinto, Tiago Loureiro Figaro da Costa and Regner, Daniel Juchem},
  title     = {UE4-Stereo: A Photorealistic Synthetic Stereo Dataset with Ground-Truth Depth and 6-DoF Poses},
  year      = {2026},
  publisher = {Hugging Face},
  doi       = {10.57967/hf/8959},
  url       = {https://huggingface.co/datasets/yanmorona/UE4-Stereo}
}