Datasetpublic

Voxel51/TartanRGBT

TartanRGBT Dataset Card TartanRGBT is a hardware-synchronized RGB–thermal robotics dataset from CMU AirLab's AnyThermal project (ICRA 2026). Features co-registered stereo RGB and thermal images across indoor, urban, park, and off-road environments. This subset: 15 trajectories 5,952 timesteps 1 Hz sampling 23,808 FiftyOne samples This is a FiftyOne dataset with 5952 samples. Installation If you haven't already, install FiftyOne: pip install -U fiftyone… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/TartanRGBT.

sourceHugging Faceupdated 4mo agoView on Hugging Face

2likes3kdownloads

Dataset Card

TartanRGBT Dataset Card

[image]

TartanRGBT is a hardware-synchronized RGB–thermal robotics dataset from CMU AirLab's AnyThermal project (ICRA 2026). Features co-registered stereo RGB and thermal images across indoor, urban, park, and off-road environments.

This subset:

—15 trajectories
—5,952 timesteps
—1 Hz sampling
—23,808 FiftyOne samples

This is a FiftyOne dataset with 5952 samples.

Installation

If you haven't already, install FiftyOne:

bash

pip install -U fiftyone

Usage

python

import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub

# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("Voxel51/TartanRGBT")

# Launch the App
session = fo.launch_app(dataset)

Dataset Sources

—Source: theairlabcmu/TartanRGBT
—Paper: arXiv:2602.06203
—License: BSD-3-Clause-Clear

Data Streams

Each timestep provides 4 synchronized camera streams:

Stream	Resolution	Notes
RGB in thermal frame	640 × 512	ZED RGB reprojected to thermal grid. Pixel-aligned with left thermal. Primary RGB–thermal pair.
Left thermal	640 × 512	FLIR Boson 640+, 8-bit grayscale
Right thermal	640 × 512	FLIR Boson 640+, 8-bit grayscale
ZED left RGB	960 × 540	Native rectified stereo
ZED right RGB	960 × 540	Native rectified stereo
Stereo depth	960 × 540	Dense depth map (meters), aligned with ZED left

Scenes

15 scenes across 5 collection days covering indoor, urban, park, and off-road terrain.

FiftyOne Structure

—Type: Grouped dataset
—Default slice: rgb_in_thermal
—Groups: 5,952

Key Fields

Field	Type	Description
`day`	str	`"day1"` – `"day5"`
`scene_name`	str	e.g. `"park_frick_seq_1_riverview_trail"`
`frame_id`	int	Frame index (0, 10, 20, …)
`timestamp`	float	Unix time (seconds)
`ffc_dropped`	bool	Exclude from training if `True` (thermal calibration event)
`pose_x/y/z`	float	Position (meters, from stereo odometry)
`pose_qx/qy/qz/qw`	float	Orientation (quaternion)

Labels

—`thermal` (rgb_in_thermal slice): Heatmap overlay of thermal intensity on RGB
—`depth` (zed_left slice): Display-optimized depth (masked, percentile-normalized)
—`depth_gt` (zed_left slice): Raw depth visualization (min/max normalized)

Use Cases

✅ Intended:

—RGB–thermal representation learning & knowledge distillation
—Cross-modal place recognition
—Monocular thermal depth estimation
—Multi-environment thermal features

❌ Out of scope:

—Odometry or metric depth benchmarking (stereo-derived, not ground truth)
—GPS-based localization

Citation

bibtex

@misc{maheshwari2026anythermallearninguniversalrepresentations,
  title={AnyThermal: Towards Learning Universal Representations for Thermal Perception},
  author={Parv Maheshwari and Jay Karhade and Yogesh Chawla and Isaiah Adu and Florian Heisen
          and Andrew Porco and Andrew Jong and Yifei Liu and Santosh Pitla
          and Sebastian Scherer and Wenshan Wang},
  year={2026},
  eprint={2602.06203},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}