HY-LiYihan/astribot_scene_replay_120eps

Astribot Scene Replay 120 Episodes

This dataset contains fixed Astribot pick-and-place replay trajectories rendered in Isaac Sim 5.1 ready scenes. The robot action sequence is fixed, while the scene and task placement vary across episodes.

Dataset Summary

• —Episodes: 120
• —Frames per episode: 1920
• —Total frames: 230400
• —FPS: 30
• —Duration per episode: 64 seconds
• —Format: LeRobot v3-style local dataset
• —Robot: Astribot S1 in Isaac Sim 5.1
• —Source replay: p&p0430_episode_1.hdf5
• —State/action source: reconstructed from replay HDF5 joint arrays
• —Video source: offline Isaac Sim 5.1 RGB camera recordings

Cameras

The dataset provides three RGB video observations:

• —observation.images.head_rgbd: 1280 x 720, 30 FPS
• —observation.images.left_wrist_rgbd: 640 x 360, 30 FPS
• —observation.images.right_wrist_rgbd: 640 x 360, 30 FPS

Low-Dimensional Features

Each frame includes:

• —observation.state: 22-dimensional Astribot joint vector
• —action: 22-dimensional Astribot command vector
• —timestamp
• —frame_index
• —episode_index
• —index
• —task_index

The 22-dimensional control/state order is:

1. 1.torso joints 1-4
2. 2.head joints 1-2
3. 3.left arm joints 1-7
4. 4.left gripper joint L1
5. 5.right arm joints 1-7
6. 6.right gripper joint L1

Important Notes

The source run was collected with video-only output, so the LeRobot observation.state and action arrays are reconstructed from the original replay HDF5:

• —observation.state: joints_dict/joints_position_state
• —action: joints_dict/joints_position_command

The videos are the actual rendered Isaac Sim outputs from the replay run. The command sequence uses the replay timing from the source metadata:

• —replay rate: 30 Hz
• —replay speed scale: 0.2
• —camera FPS: 30
• —gripper lead: 0 seconds
• —initial pose: ready pose

Directory Layout

scene/
scene/ready/
meta/info.json
meta/stats.json
meta/tasks.parquet
meta/episodes/chunk-000/file-000.parquet
data/chunk-000/file-000.parquet
videos/observation.images.head_rgbd/chunk-000/file-000.mp4
videos/observation.images.left_wrist_rgbd/chunk-000/file-000.mp4
videos/observation.images.right_wrist_rgbd/chunk-000/file-000.mp4

The scene/ folder contains the original USD scene files. The scene/ready/ folder contains the processed USDA versions used by the replay dataset collection pipeline.

Intended Use

This dataset is intended for visual policy debugging, replay-conditioned imitation learning experiments, and validating scene-level visual generalization for Astribot manipulation tasks.

Loading With LeRobot

The dataset has been smoke-tested with lerobot==0.4.4.

from lerobot.datasets.lerobot_dataset import LeRobotDataset

ds = LeRobotDataset(
    repo_id="HY-LiYihan/astribot_scene_replay_120eps",
    video_backend="pyav",
)

item = ds[0]
print(len(ds))
print(item["observation.images.head_rgbd"].shape)
print(item["observation.state"].shape)
print(item["action"].shape)

video_backend="pyav" is recommended for this dataset on machines where TorchCodec cannot find system FFmpeg shared libraries.