HRDexDB/HRDexDB-DTW-Action-Synced-224

HRDexDB DTW Action-Synced 224

This dataset contains temporally aligned human–robot episode pairs derived from HRDexDB.

Contents

• —Human and robot videos aligned with object-trajectory Dynamic Time Warping (DTW)
• —224×224 RGB video encoded as H.264 at 30 FPS
• —Robot arm and hand actions synchronized to every output frame
• —Human MANO vertices, shared topology, parameters, and joints synchronized to every output frame
• —Per-pair manifests preserving source frame IDs and provenance

Robot actions use zero-order hold at the mapped robot video timestamp. Video resizing uses CUDA torch.nn.functional.interpolate with bilinear interpolation, align_corners=False, and antialiasing enabled.

For episodes whose timestamp stream ends a few frames before an otherwise valid video/pose stream, up to eight trailing timestamps are extrapolated using the median positive interval of the last 100 timestamps. robot_action.npz records the per-frame timestamp_extrapolated mask, timestamp_extrapolation_count, and timestamp_extrapolation_dt used by this bounded recovery policy.

Layout

<human-robot-target>/<object>/robot_<robot_episode>__human_<human_episode>/
├── manifest.json
├── robot_action.npz
├── human_mano.npz
└── views/<camera_id>/
    ├── human.mp4
    └── robot.mp4

Active generation targets are human-inspire_f1 and human_inspire_dftp. Generation of human-allegro_v5 was stopped by user request; its already-completed pairs remain in the dataset and no existing output was removed.

Earlier completed pairs retain all 22–23 common camera views. After the view-policy change recorded in run_manifest.json, newly generated pairs contain these three views when available:

• —22645021
• —23022639
• —23263780

run_manifest.json and the per-target index.json files contain final completion counts and processing settings. failed_pairs.json records pairs that could not be exported, such as episodes with missing robot action streams.

Array conventions

robot_action.npz contains output_frame, human_frame, robot_frame, robot_timestamp, arm_action_source_index, hand_action_source_index, arm_action, and hand_action.

human_mano.npz contains output_frame, human_frame, vertices, shared faces, global_orient, hand_pose, betas, transl, joints, and fitting loss.

The original /media/hdd1/hrdexdb dataset was treated as read-only during export.