kinam0252/merged_strawberries_tictactoe

SO101 Merged Strawberries & Tic-Tac-Toe Dataset

Dataset Description

This dataset contains robot manipulation episodes combining two distinct tasks using SO101 robot. The dataset includes 27 episodes of robot manipulation with multi-view camera observations.

Tasks

Task 0: Pick up strawberries and put them in a tray Task 1: Pick up an X piece and place it on the tic-tac-toe board to complete the winning line

Robot: SO101 Follower robot with 6 degrees of freedom

Dataset Structure

This dataset follows the LeRobot format and contains:

• —Data: Parquet files containing robot states, actions, and metadata
• —Videos: MP4 files with camera observations (front and wrist views)
• —Metadata: JSON files with episode information and statistics

Task Distribution

Features

• —Action: 6-dimensional robot joint positions
• —shoulder_pan.pos: Shoulder pan joint position
• —shoulder_lift.pos: Shoulder lift joint position
• —elbow_flex.pos: Elbow flex joint position
• —wrist_flex.pos: Wrist flex joint position
• —wrist_roll.pos: Wrist roll joint position
• —gripper.pos: Gripper position

• —Observation State: 6-dimensional robot joint states (same as action)

• —Images: Multi-view camera observations
• —observation.images.front: Front camera view (256x256x3 RGB)
• —observation.images.wrist: Wrist camera view (256x256x3 RGB)

• —Metadata:
• —timestamp: Frame timestamps in seconds
• —frame_index: Frame index within episode
• —episode_index: Episode index
• —task_index: Task index (0 for strawberry picking, 1 for tic-tac-toe)

Usage

Using LeRobot

from lerobot import load_dataset
# Load the dataset
dataset = load_dataset("kinam0252/merged_strawberries_tictactoe")
# Access episodes by task
strawberry_episodes = dataset.filter(lambda x: x["task_index"] == 0)
tic_tac_toe_episodes = dataset.filter(lambda x: x["task_index"] == 1)
# Access episodes
for episode in dataset:
    print(f"Episode {episode['episode_index']}: {len(episode)} frames")
    print(f"Task: {episode['task_index']}")
    print(f"Actions shape: {episode['action'].shape}")
    print(f"Images keys: {list(episode['observation.images'].keys())}")

Direct Loading

import pandas as pd
import json
# Load episode data
df = pd.read_parquet("data/chunk-000/episode_000000.parquet")
# Load metadata
with open("meta/info.json", "r") as f:
    info = json.load(f)
# Access robot actions and states
actions = df["action"].values
states = df["observation.state"].values
# Filter by task
strawberry_data = df[df["task_index"] == 0]
tic_tac_toe_data = df[df["task_index"] == 1]

Dataset Statistics

• —Total Episodes: 27
• —Total Frames: 46375
• —Total Tasks: 2
• —FPS: 30
• —Average Episode Length: ~1717 frames (~57 seconds)
• —Robot DOF: 6 (shoulderpan, shoulderlift, elbowflex, wristflex, wrist_roll, gripper)
• —Video Resolution: 256x256x3 RGB
• —Video Codec: AV1

Citation

If you use this dataset, please cite:

@dataset{merged_strawberries_tictactoe_2024,
  title={SO101 Merged Strawberries & Tic-Tac-Toe Dataset},
  author={Your Name},
  year={2024},
  publisher={Hugging Face},
  url={https://huggingface.co/datasets/kinam0252/merged_strawberries_tictactoe}
}

License

This dataset is licensed under the MIT license.

Acknowledgments

• —Built using the LeRobot framework
• —Robot demonstrations performed with SO101 robot arm
• —Video encoding optimized with AV1 codec for efficient storage
• —Merged from two distinct manipulation tasks for multi-task learning