CoolFace
Datasetpublic

k1000dai/libero_subtask_sarm

This dataset was created using LeRobot. SARM Subtask Annotations This is the LIBERO dataset (1693 episodes, 40 tasks) augmented with SARM dense_only subtask annotations. Each episode's high-level task instruction is decomposed into ordered atomic subtasks (from decompositions.json), and a vision-language model (Qwen/Qwen3-VL-8B-Instruct) localizes each subtask in time. Per-episode columns added to meta/episodes/*.parquet: dense_subtask_names, dense_subtask_start_times… See the full description on the dataset page: https://huggingface.co/datasets/k1000dai/libero_subtask_sarm.

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
0likes183downloads
Dataset Card

This dataset was created using LeRobot.

SARM Subtask Annotations

This is the LIBERO dataset (1693 episodes, 40 tasks) augmented with [SARM](https://huggingface.co/docs/lerobot/sarm) dense_only subtask annotations. Each episode's high-level task instruction is decomposed into ordered atomic subtasks (from decompositions.json), and a vision-language model (Qwen/Qwen3-VL-8B-Instruct) localizes each subtask in time.

Per-episode columns added to meta/episodes/*.parquet:

  • —dense_subtask_names, dense_subtask_start_times, dense_subtask_end_times, dense_subtask_start_frames, dense_subtask_end_frames — the fine-grained subtasks and their boundaries.
  • —sparse_subtask_* (and legacy unprefixed subtask_*) — a single auto-generated "task" stage spanning the full episode.

Dataset-level priors in meta/:

  • —temporal_proportions_sparse.json — {"task": 1.0}.
  • —temporal_proportions_dense.json — average temporal proportion per subtask name across the dataset.

Notes: 1674 episodes use VLM-localized boundaries; ~48 (the duplicate-subtask task "put both moka pots on the stove", plus a few where the VLM produced an off-count segmentation or failed) use a uniform-time split over the canonical subtask names. All dense subtask names match the decompositions.json vocabulary exactly. Train with lerobot-train --policy.type=sarm --policy.annotation_mode=dense_only --policy.image_key=observation.images.image.

Per-frame subtask_index (dataset_subtask format)

In addition to the SARM episode-level columns above, this dataset follows the Using Subtasks format:

  • —meta/subtasks.parquet — maps each of the 52 unique subtask strings to a subtask_index (same layout as tasks.parquet).
  • —A per-frame subtask_index column in data/chunk-*/file-*.parquet (added to info.json features) — each frame is labeled with the dense subtask active at that frame (the most-recently-started subtask; idle trailing frames take the last subtask).
python
from lerobot.datasets import LeRobotDataset
ds = LeRobotDataset("k1000dai/libero_subtask_sarm")  # or the local path
sample = ds[100]
sample["subtask_index"]  # tensor(2)

Note: resolving subtask_index → the natural-language subtask string at load time (dataset.meta.subtasks, sample["subtask"]) requires the LeRobot version documented in the page above; the on-disk format here is conformant and the subtask_index column loads on the current version.

Dataset Description

  • —Homepage: [More Information Needed]
  • —Paper: [More Information Needed]
  • —License: apache-2.0

Dataset Structure

meta/info.json:

json
{
    "codebase_version": "v3.0",
    "robot_type": "panda",
    "total_episodes": 1693,
    "total_frames": 273465,
    "total_tasks": 40,
    "chunks_size": 1000,
    "fps": 10.0,
    "splits": {
        "train": "0:1693"
    },
    "data_path": "data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet",
    "video_path": "videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4",
    "features": {
        "observation.images.image": {
            "dtype": "image",
            "shape": [
                256,
                256,
                3
            ],
            "names": [
                "height",
                "width",
                "channel"
            ],
            "fps": 10.0
        },
        "observation.images.image2": {
            "dtype": "image",
            "shape": [
                256,
                256,
                3
            ],
            "names": [
                "height",
                "width",
                "channel"
            ],
            "fps": 10.0
        },
        "observation.state": {
            "dtype": "float32",
            "shape": [
                8
            ],
            "names": [
                "state"
            ],
            "fps": 10.0
        },
        "action": {
            "dtype": "float32",
            "shape": [
                7
            ],
            "names": [
                "actions"
            ],
            "fps": 10.0
        },
        "timestamp": {
            "dtype": "float32",
            "shape": [
                1
            ],
            "names": null,
            "fps": 10.0
        },
        "frame_index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null,
            "fps": 10.0
        },
        "episode_index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null,
            "fps": 10.0
        },
        "index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null,
            "fps": 10.0
        },
        "task_index": {
            "dtype": "int64",
            "shape": [
                1
            ],
            "names": null,
            "fps": 10.0
        }
    },
    "data_files_size_in_mb": 100,
    "video_files_size_in_mb": 500
}

Citation

BibTeX:

bibtex
[More Information Needed]