Project-IRA/TPSoSe2026_Dataset_Fetch_Ball_LeRobot_SO101

SO-101 Fetch Ball — 100 episodes

Teleoperated SO-101 demonstrations of picking up a ball and placing it into a waiting human hand. The smallest and most recently recorded task dataset in Project-IRA.

Part of [Project-IRA](https://huggingface.co/Project-IRA) — Interactive Robotic Arm. Code: https://github.com/Project-IRA/interactive-robotic-arm

Composition

100 episodes across 10 prompt phrasings, 10 episodes each.

Recording scheme: a human holds their hand in one position while the ball is placed on the table; the arm picks up the ball and places it into the waiting hand. The hand position is held constant for 5 episodes at a time, then moved.

This is the only task in Project-IRA involving direct human-robot handover, which means a human hand is present in the camera frames throughout.

Robot setup

Inference note: both cameras are run at 640x480 during inference, not at their recording resolutions, to reduce the payload sent to the inference server.

Schema

Recording protocol

• —Recorded by teleoperating the SO-101 follower with an SO-101 leader arm.
• —10 episodes per prompt. The prompt phrasing was deliberately changed roughly every 10 episodes, so language conditioning sees many surface forms of the same intent.
• —Object positions and scene difficulty were varied systematically within each block (e.g. early episodes with a single object, later ones with several).
• —Recovery behaviour is incidental. Where the operator made a mistake mid-episode and corrected it, that correction stayed in the data. No recovery episodes were scripted deliberately, so recovery coverage is uneven.

Prompts

All 10 prompts, in English, 10 episodes each.

1. 1.Fetch the ball and put it into my hand
2. 2.Bring the ball back into my hand
3. 3.Retrieve the ball and drop it into the waiting hand
4. 4.Put the ball into the open hand
5. 5.Carry the ball and set it down in the waiting palm
6. 6.Pass the ball to the waiting hand
7. 7.Fetch the ball
8. 8.Find the ball on the desk and put it in the hand
9. 9.Grab the multicolored ball and then drop it gently into the hand
10. 10.Pick up the round ball and place it into the hand

Usage

from lerobot.datasets.lerobot_dataset import LeRobotDataset

ds = LeRobotDataset("Project-IRA/TPSoSe2026_Dataset_Fetch_Ball_LeRobot_SO101")
print(ds.meta.info)

Train a policy on it:

lerobot-train \
    --policy.path=lerobot/smolvla_base \
    --dataset.repo_id=Project-IRA/TPSoSe2026_Dataset_Fetch_Ball_LeRobot_SO101 \
    --batch_size=64 --steps=200000 \
    --policy.device=cuda

Models trained on this dataset

No single-task model was trained on this dataset. Ball fetching is covered by the multi-task models trained on `Dataset_Full_Merged_Final_V1` — best results from Pi05 V7 Full V2.

[!NOTE] Because this task involves handing an object to a person, human hands appear in the training frames. If you fine-tune on this data, be aware the policy will move toward a human hand in the workspace by design.

Limitations

• —Single environment. One desk, one lighting setup, one camera geometry, one set of physical objects. Policies trained here should not be expected to transfer.
• —Teleoperated demonstrations vary in quality and speed between operators and sessions.
• —Recovery coverage is uneven — see the recording protocol note above.
• —No held-out split. The dataset ships as a single train split; all episodes were used for training. Evaluation was done by running policies on the physical arm.

Licensing

Released under CC BY-SA 4.0. This is a share-alike licence: you may use, share and adapt this dataset, including commercially, provided you give attribution and release any derivative dataset under the same licence. It cannot be taken closed-source.

Recorded with LeRobot (Apache-2.0); the LeRobot dataset format and tooling remain under their original licence.

Citation

@misc{project_ira_2026,
  title        = {Project-IRA: Interactive Robotic Arm},
  author       = {Baten, Cleo and Keppler, Bela and Sapper, Jonas},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Project-IRA}},
  note         = {Code: \url{https://github.com/Project-IRA/interactive-robotic-arm}}
}