ntnu-arl/unified_autonomy_stack_datasets

1---2license: bsd-3-clause3---4 5# Unified Autonomy Stack Datasets6 7This repository contains data released in relation with the Unified Autonomy Stack. Details can be found here8 9- Paper: <https://arxiv.org/abs/2605.12735>10- Code: <https://ntnu-arl.github.io/unified_autonomy_stack/>11 12The data was collected using the following platforms in both manually piloted and autonomously operated modes:13 14- **AR-1 (Hornbill):** A variant of the [RMF-Owl](https://ieeexplore.ieee.org/document/9836115) collision-tolerant aerial robot.15- **AR-2 (Magpie):** A collision-tolerant aerial robot designed to carry the UniPilot module.16- **GR-1 (Anymal):** An ANYmal D quadruped robot from ANYbotics carrying the UniPilot module.17- **UniPilot:** A compact hardware-software autonomy payload that can be integrated across diverse robot embodiments to enable autonomous operation in GPS-denied environments. In addition to being carried by AR-2 and GR-1, a handheld (helmet-mounted) variant was used for the `campus_fog` sequence.18 19![Platforms](assets/platforms.png)20 21The following table summarizes the platforms used for each dataset:22 23|  Name | Location (Norway) | Platform(s) | Notes |24|--------------|----------|-------------|-------------|25| basement/nmpc | NTNU Elektro Building basement | AR-2 (Magpie) | Autonomous Full Stack (Navigation mode: NMPC) |26| basement/rl | NTNU Elektro Building basement | AR-2 (Magpie) | Autonomous Full Stack (Navigation mode: RL) |27| elektro_hall | NTNU Elektro Building | GR-1 (UniPilot-Anymal) | Autonomous Full Stack |28| løkken_mine | Løkken mine | AR-2 (Magpie) | Autonomous Full Stack |29| runehamar_tunnel/magpie | Runehamar tunnel | AR-2 (Magpie) | Autonomous Full Stack |30| runehamar_tunnel/hornbill | Runehamar tunnel | AR-1 (Hornbill) | Manually Piloted, SLAM Evaluation |31| frozen_lake | Jonsvatnet Lake | AR-1 (Hornbill) | Manually Piloted, SLAM Evaluation |32| fyllingsdalen_tunnel | Fyllingsdalen tunnel | AR-1 (Hornbill) | Manually Piloted, SLAM Evaluation |33| campus_fog | NTNU campus | UniPilot | Helmet-mounted walking, SLAM Evaluation |34 35## Sensor Setup36 37| Sensor | AR-1 (Hornbill) | UniPilot (AR-2 / GR-1 / handheld) |38|:------:|:----------------|:----------------------------------|39| LiDAR  | [Ouster OS0-128 Rev7](https://ouster.com/products/hardware/os0-lidar-sensor) | [RoboSense Airy](https://www.robosense.ai/en/rslidar/Airy); [Hesai JT-128](https://www.hesaitech.com/product/jt128/) on the handheld `campus_fog` |40| Camera | [FLIR Blackfly S 0.4 MP Color](https://www.teledynevisionsolutions.com/en-150/products/blackfly-s-usb3/?model=BFS-U3-04S2C-C&vertical=machine%20vision&segment=iis) | 3× [MIPI Vision Components IMX296-C](https://www.mipi-modules.com/en/mipi-camera-modules-technical-data/) |41| Radar  | [TI IWR6843AOP](https://www.ti.com/tool/IWR6843AOPEVM) | [D3 Embedded RS-6843AOPU FMCW](https://www.d3embedded.com/product/designcore-rs-6843aopu-mmwave-radar-sensor/); [uRAD Industrial](https://urad.es/en/product/urad-radar-industrial/) on the handheld `campus_fog` |42| IMU    | [VectorNav VN-100](https://www.vectornav.com/products/detail/vn-100) | [VectorNav VN-100](https://www.vectornav.com/products/detail/vn-100) |43| Compute | [Khadas VIM4](https://www.khadas.com/vim4) | [NVIDIA Jetson Orin NX](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/) |44 45## Data Description46 47Each sequence is released as a **ROS 1 bag** named `sensors_only.bag` containing the raw sensor data collected during the run. The ground truth is provided as a `.tum` file for the sequences where it is available.48 49Due to resource constraints on the platforms, the LiDAR is recorded as **raw packets** instead of the deserialized point clouds. To obtain bags with the point clouds, please view the [lidar_packets_to_pointclouds.md](lidar_packets_to_pointclouds.md).50 51### Topics52 53The radar, IMU, and static transforms are common to all platforms; only the LiDAR and camera topics differ. Bags may additionally contain: flight-controller telemetry under `/mavros/*` (including GNSS as `sensor_msgs/NavSatFix` and `mavros_msgs/GPSRAW` where a fix was available), hardware time-synchronization triggers and status under `/sensor_sync_node/*`, further VN-100 streams (temperature, filtered IMU), and per-sensor host-receive timestamps under `/<sensor>/ros_time_now`.54 55#### Common to all platforms56 57| Sensor | Topic | Datatype | Rate |58| --- | --- | --- | --- |59| Radar | `/radar/cloud` | `sensor_msgs/PointCloud2` | 10 Hz |60| IMU | `/vectornav_driver_node/imu/data` | `sensor_msgs/Imu` | 200 Hz |61| Magnetometer | `/vectornav_driver_node/imu/mag` | `sensor_msgs/MagneticField` | 200 Hz |62| Barometer | `/vectornav_driver_node/pressure` | `sensor_msgs/FluidPressure` | 200 Hz |63| Extrinsics (static transforms) | `/tf_static` | `tf2_msgs/TFMessage` | — |64 65#### LiDAR66 67Recorded as raw packets corresponding to a 10 Hz point cloud (one LiDAR per platform):68 69| Platform (LiDAR) | Packets | LiDAR IMU | Other |70| --- | --- | --- | --- |71| AR-1 (Ouster OS0-128) | `/ouster/lidar_packets` (`ouster_ros/PacketMsg`) | `/ouster/imu_packets` (`ouster_ros/PacketMsg`, 100 Hz) | `/ouster/metadata` (`std_msgs/String`) |72| UniPilot AR-2 / GR-1 (RoboSense Airy) | `/rslidar_packets` (`rslidar_msg/RslidarPacket`) | `/rslidar_imu_data` (`sensor_msgs/Imu`, 200 Hz) | — |73| UniPilot handheld (Hesai JT-128) | `/lidar_packets` (`hesai_ros_driver/UdpFrame`) | `/lidar_imu` (`sensor_msgs/Imu`) | `/lidar_packets_loss` (`hesai_ros_driver/LossPacket`) |74 75#### Cameras76 77`sensor_msgs/CompressedImage` at 20 Hz:78 79| Platform | Topic(s) |80| --- | --- |81| AR-1 (Hornbill) | `/cam0/cam0/compressed` (intrinsics on `/cam0/camera_info`) |82| UniPilot (AR-2 / GR-1 / handheld) | `/cam_front/image_raw/compressed`, `/cam_left/image_raw/compressed`, `/cam_right/image_raw/compressed` |83 84> On `fyllingsdalen_tunnel`, the AR-1 camera and radar were recorded at 25 Hz (the radar chirp configuration was changed for high-speed flight).85 86### Calibration87 88#### Intrinsics89 90Camera intrinsics are provided in the [calibration folder](calibration/), with the same intrinsics for all sequences recorded on a given platform.91 92#### Extrinsics93 94The transforms between sensors are provided below. The transform $T_{AB}$ transforms a point from frame $B$ to frame $A$ as $p_A = T_{AB} * p_B$, where the point is represented in homogeneous coordinates. All transforms are provided in the format: `x, y, z, qx, qy, qz, qw`.95 96```97AR-1 (Hornbill):98  T_imu_lidar: [0.0166, 0.02158, 0.03375, 0, 0, 0, 1] 99  T_imu_radar: [0.077, 0.016, -0.063, 0.963, -0.021, -0.265, 0.021]100  T_imu_cam: [0.0725765278611583, 0.018936068067624674, -0.03560091123164558, 0.5543417213240229, -0.5433799916950063, 0.44199772312966146, 0.4495347076402992]101 102AR-2 / GR-1 (UniPilot):103  T_imu_lidar: [-0.06605, -0.01878, 0.034, 0.707, 0.00, -0.707, 0.00]104  T_imu_radar: [0.07717380907196035, -0.0479741168664902, 0.006770362043579366, 0.01761050968654745, 0.25032180256367187, -0.017699746191127304, 0.9679631109162291]105 106UniPilot handheld:107  T_imu_lidar: [-0.11484, -0.01878, 0.035, 0.3799282, -0.5963678, -0.3799282, 0.5963678]108  T_imu_radar: [0.08373, 0.0213, 0.02691, 0.6830127, 0.1830128, -0.1830128, 0.6830127]109  T_imu_cam_front: [0.083743715, 0.000235985, 0.005836153, -0.608415182, 0.616574414, -0.353450887, 0.353184694]110  T_imu_cam_left: [-0.021764534, 0.045545798, 0.023474280, -0.706455147, 0.000556152, -0.002366216, 0.707753642]111  T_imu_cam_right: [-0.025516567, -0.089169163, 0.024118153, 0.008477527, 0.715010265, -0.699047196, -0.004633580]112 113```114 115## Ground Truth116 117Where available, ground truth is provided as a [TUM-format](https://vision.in.tum.de/data/datasets/rgbd-dataset/file_formats) trajectory file (`timestamp tx ty tz qx qy qz qw`) alongside the bag.118 119- **Tunnels (`fyllingsdalen_tunnel`, `runehamar_tunnel/hornbill`):** generated by fusing the tracking of a Leica GRZ101 mini-prism (mounted on AR-1) by a Leica MS60 MultiStation with the onboard IMU, in an offline Levenberg-Marquardt optimization.120- **`campus_fog` and `frozen_lake`:** GNSS was available, so ground truth is created using a GNSS-augmented visual bundle adjustment optimization with Pix4DMatic.121 122## Citation123 124If you use this data in your research, please cite the following publication:125 126```127@misc{dharmadhikari2026unifiedautonomystackblueprint,128      title={The Unified Autonomy Stack: Toward a Blueprint for Generalizable Robot Autonomy}, 129      author={Mihir Dharmadhikari and Nikhil Khedekar and Mihir Kulkarni and Morten Nissov and Martin Jacquet and Angelos Zacharia and Marvin Harms and Albert Gassol Puigjaner and Philipp Weiss and Kostas Alexis},130      year={2026},131      eprint={2605.12735},132      archivePrefix={arXiv},133      primaryClass={cs.RO},134      url={https://arxiv.org/abs/2605.12735}, 135}136```137