zfff/sroiv2_strawberry_picking_lab_1459_occlusion

This dataset was created using LeRobot.

<a class="flex" href="https://huggingface.co/spaces/lerobot/visualizedataset?path=zfff/sroiv2strawberrypickinglab1459occlusion"> <img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/> <img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/> </a>

SROI v2 — Strawberry Picking (Lab, Occlusion) — Training Set

[image]

Human strawberry-harvesting demonstrations for SROI V2 (Strawberry Robotic Operation Interface, Zhejiang University), collected with the SROI V2 handheld data-acquisition device — a UMI-style gripper with an integrated Intel RealSense D405 stereo camera. A human operator picks strawberries from live plants in a laboratory setup while the device records first-person video; no robot arm is involved during collection. The 7-DoF end-effector actions are recovered off-line: device trajectory from ORB-SLAM3 stereo SLAM (with the gripper masked out of the tracking frames), gripper opening from AprilTags on the gripper. The learned policy is then deployed on a robot arm carrying the same end effector, whose camera viewpoint is identical to the one in these recordings — that viewpoint match is what makes the imitation learning transfer.

• —Task: pick the strawberry (single task, all 1,459 episodes)
• —Collection: human demonstrations with the SROI V2 handheld device (UMI-style — not recorded on a robot arm)
• —Camera: Intel RealSense D405 mounted on the device (18 mm stereo baseline), 480×640 RGB, 30 fps, first-person view
• —Project: agroboticsresearch.github.io/sroi_v2 · all SROI datasets
• —Paper: arXiv:2501.16717
• —This repository is the training set. The paired held-out evaluation set is `zfff/sroiv2_strawberry_picking_lab_validation` (recorded on a different day; no leakage).

Quickstart

from lerobot.datasets.lerobot_dataset import LeRobotDataset

dataset = LeRobotDataset("zfff/sroiv2_strawberry_picking_lab_1459_occlusion")
episode = dataset[0]  # dict with "observation.images.camera", "action", ...

Dataset Summary

Composition / Provenance

Merged (with the official lerobot_edit_dataset merge tool) from two collections that share identical schema, fps, and codec:

All episodes passed visual QC (ok rating) before conversion. The gripper position channel is normalized to a single shared range recovered from the 1000-onesb training recordings (gripper_min_distance=84.079, gripper_max_distance=261.780, recorded in meta/gripper_normalization.json), so action[6] is directly comparable across every episode.

Dataset Structure

meta/info.json:

{
    "codebase_version": "v3.0",
    "fps": 30,
    "features": {
        "observation.images.camera": {
            "dtype": "video",
            "shape": [480, 640, 3],
            "names": ["height", "width", "channels"],
            "info": {
                "video.height": 480,
                "video.width": 640,
                "video.codec": "av1",
                "video.pix_fmt": "yuv420p",
                "video.is_depth_map": false,
                "video.fps": 30,
                "video.channels": 3,
                "has_audio": false,
                "video.video_backend": "pyav"
            }
        },
        "action": {
            "dtype": "float32",
            "names": ["ee.x", "ee.y", "ee.z", "ee.wx", "ee.wy", "ee.wz", "ee.gripper_pos"],
            "shape": [7]
        },
        "timestamp": {"dtype": "float32", "shape": [1], "names": null},
        "frame_index": {"dtype": "int64", "shape": [1], "names": null},
        "episode_index": {"dtype": "int64", "shape": [1], "names": null},
        "index": {"dtype": "int64", "shape": [1], "names": null},
        "task_index": {"dtype": "int64", "shape": [1], "names": null}
    },
    "total_episodes": 1459,
    "total_frames": 140522,
    "total_tasks": 1,
    "chunks_size": 1000,
    "data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
    "video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4",
    "robot_type": "so100",
    "splits": {"train": "0:1459"}
}

Note: robot_type: "so100" in meta/info.json is a hardcoded default of the conversion script (sroi_to_lerobot.py) and does not describe the collection rig — this data is human-collected with the SROI V2 handheld device.

Citation

If you use this dataset, you are welcome to cite:

Hou, L., Lu, W., Wang, Y., Peng, C., & Fei, Z. (2025). Strawberry Robotic Operation Interface: An Open-Source Device for Collecting Dexterous Manipulation Data in Robotic Strawberry Cultivation. IFAC-PapersOnLine, 59(23), 303–308. arXiv:2501.16717.

License

Apache-2.0