THULab/cmu_stretch

CMU Stretch TsFile

This dataset is a TsFile conversion of the Hugging Face dataset `lerobot/cmu_stretch`, created using LeRobot.

Dataset Description

• —Original dataset: `lerobot/cmu_stretch`
• —Homepage: https://robo-affordances.github.io/
• —Paper: https://arxiv.org/abs/2304.08488
• —License: MIT
• —Episodes: 135
• —Frames: 25,016
• —Tasks: 5
• —Sampling rate: 5 fps
• —Split: train (0:135)

TsFile Conversion

The numeric robot state, action, reward, done, time, and task fields are stored in one TsFile table:

• —File: data/cmu_stretch.tsfile
• —Table: cmu_stretch
• —Time column: Time, computed as round(timestamp * 1000) in milliseconds
• —TAG columns: episode_index, task_index
• —Rows: 25,016
• —Granularity: all train episodes merged into one TsFile table

Vector source columns are flattened into scalar TsFile FIELD columns:

• —observation.state -> observation_state_0 ... observation_state_3
• —action -> action_0 ... action_7

Scalar source columns are normalized as follows:

• —next.reward -> next_reward
• —next.done -> next_done
• —index -> sample_index
• —frame_index is kept unchanged

The source timestamp column is dropped because it is represented by Time / 1000 seconds after conversion.

Video Policy

Camera videos are not included in this TsFile repository. Video data remains in the original Hugging Face dataset:

https://huggingface.co/datasets/lerobot/cmu_stretch/tree/main/videos

The converted meta/info.json keeps the original video path under video_path_original and documents the conversion in tsfile_conversion.

Citation

@inproceedings{bahl2023affordances,
    title={Affordances from Human Videos as a Versatile Representation for Robotics},
    author={Bahl, Shikhar and Mendonca, Russell and Chen, Lili and Jain, Unnat and Pathak, Deepak},
    booktitle={CVPR},
    year={2023}
}

@article{mendonca2023structured,
    title={Structured World Models from Human Videos},
    author={Mendonca, Russell and Bahl, Shikhar and Pathak, Deepak},
    journal={CoRL},
    year={2023}
}