Datasetpublic

ITHwangg/svla_koch_pickplace_v3

Dataset Overview NOTE: The episode_106 and episode_122 ~ 125 were made incorrectly. They should be ignored when training the SmolVLA model. The dataset was created by the team Lebotica during LeRobot Worldwide Hackathon and used for training the SmolVLA model on structured robotic manipulation prompts The dataset consists of 122 tasks and 1 instruction, and there are the two types of episodes: episode_0 ~ episode_53: Pick a color ball among the balls scattered on the white… See the full description on the dataset page: https://huggingface.co/datasets/ITHwangg/svla_koch_pickplace_v3.

sourceHugging Facemitupdated 1y agoView on Hugging Face

0likes287downloads

Dataset Card

Dataset Overview

NOTE: The episode106 and episode122 ~ 125 were made incorrectly. They should be ignored when training the SmolVLA model.

—The dataset was created by the team Lebotica during LeRobot Worldwide Hackathon and used for training the SmolVLA model on structured robotic manipulation prompts
—The dataset consists of 122 tasks and 1 instruction, and there are the two types of episodes:
—episode0 ~ episode53: Pick a color ball among the balls scattered on the white plate and place it in the corresponding color plate.
—episode54 ~ episode121: Pick a color ball among the 9 balls placed at the fixed positions in the white plate and place it in the corresponding color plate.
—You can check the demo of the trained SmolVLA in the Hackathon Demo Page (Team number: 76).
—This dataset is also shared in LeRobot-worldwide-hackathon/76-Lebotica-Pick_with_Color_Matching_and_Place_into_Plates

Dataset Structure

├── data
│   └── chunk-000
│       ├── episode_000000.parquet
│       ├── ...
│       └── episode_000121.parquet
├── meta
│   ├── episodes.jsonl
│   ├── episodes_stats.jsonl
│   ├── info.json
│   └── tasks.jsonl
└── videos
    └── chunk-000
        ├── observation.images.side
        │   ├── episode_000000.mp4
        │   ├── ...
        │   └── episode_000121.mp4
        └── observation.images.top
            ├── episode_000000.mp4
            ├── ...
            └── episode_000121.mp4

—The tasks.json file contains an array of 122 task prompts. Each prompt follows a structured template for robotic manipulation.
—Example prompt:

text

   Pick a (red | blue | green) ball from the (top | middle | bottom)-(left | center | right) and place in the (red | blue | green) plate.

Usage

To use this dataset for training SmolVLA:

1.First, install the required dependencies:

bash

   git clone https://github.com/huggingface/lerobot.git
   cd lerobot
   pip install -e ".[smolvla]"

1.Train SmolVLA

bash

   python lerobot/scripts/train.py \
    --dataset.repo_id=ITHwangg/svla_koch_pickplace_v2 \
    --policy.path=lerobot/smolvla_base \
    --num_workers=8 \
    --batch_size=64 \
    --steps=100000 \
    --eval_freq=500 \
    --log_freq=10 \
    --save_freq=500 \
    --save_checkpoint=true

1.Caution
2.Currently, the python script refers to the branch named v2.1.
3.Every data/chunk-000/*.parquet has only the task index 0 so you should map epicode indexes to task indexes one by one:

python

     # lerobot/lerobot/common/datasets/lerobot_dataset.py
     class LeRobotDataset(torch.utils.data.Dataset):
        def __init__(
            self,
            repo_id: str,
            root: str | Path | None = None,
            episodes: list[int] | None = None,
            image_transforms: Callable | None = None,
            delta_timestamps: dict[list[float]] | None = None,
            tolerance_s: float = 1e-4,
            revision: str | None = None,
            force_cache_sync: bool = False,
            download_videos: bool = True,
            video_backend: str | None = None,
        ):
     
        ...
     
        # Load actual data
        try:
            if force_cache_sync:
                raise FileNotFoundError
            assert all((self.root / fpath).is_file() for fpath in self.get_episodes_file_paths())
            self.hf_dataset = self.load_hf_dataset()
        except (AssertionError, FileNotFoundError, NotADirectoryError):
            self.revision = get_safe_version(self.repo_id, self.revision)
            self.download_episodes(download_videos)
            self.hf_dataset = self.load_hf_dataset()
     
        # HERE ###########################
        # After loading the dataset and setting up episode_data_index
        if self.hf_dataset is not None:
            # Create a new column with task_index = episode_index
            new_task_index = torch.stack(self.hf_dataset["episode_index"])
            self.hf_dataset = self.hf_dataset.map(
                lambda x, idx: {"task_index": new_task_index[idx]}, with_indices=True
            )
        ##################################

        self.episode_data_index = get_episode_data_index(self.meta.episodes, self.episodes)
     
        ...

License

This dataset is released under the MIT License.