Hoshipu/libero-logic

LIBERO Logical State and Action Trajectories

This repository contains LIBERO robot manipulation trajectories augmented with per-frame logical state and logical action annotations.

The data is stored as HDF5 files under datasets/. Each suite has one directory, and each task has one HDF5 file containing multiple demonstrations.

Recommended Hugging Face Layout

Keep the repository organized like this:

.
├── README.md
├── requirements.txt
├── visualize_dataset.py
└── datasets/
    ├── libero_10/
    │   └── *_demo.hdf5
    ├── libero_90/
    │   └── *_demo.hdf5
    ├── libero_goal/
    │   └── *_demo.hdf5
    ├── libero_object/
    │   └── *_demo.hdf5
    └── libero_spatial/
        └── *_demo.hdf5

Use Git LFS for the HDF5 data files. The included .gitattributes marks *.hdf5 files for LFS storage.

Hugging Face can host this layout directly as a dataset repository. Because the trajectories are HDF5 robot data rather than CSV/JSONL/Parquet rows, the Hub's automatic dataset viewer may not render the contents directly. Use visualize_dataset.py for interactive inspection.

Dataset Contents

After filtering trajectories without logical annotations, the dataset contains:

Each HDF5 task file follows this structure:

data/
  demo_0/
    actions
    dones
    rewards
    robot_states
    states
    logical_actions
    logical_states
    obs/
      agentview_rgb
      eye_in_hand_rgb
      ee_pos
      ee_ori
      ee_states
      gripper_states
      joint_states
    logical/
      frame_indices
      states
      actions
  demo_1/
  ...

The top-level data group also stores task metadata in HDF5 attributes such as problem_info, env_args, env_name, num_demos, and total.

Per-Frame Schema

For a frame index i in data/demo_N, the aligned per-frame fields are:

The logical/ subgroup preserves the original synced logical arrays:

logical_states and logical_actions at the demo root are convenience arrays with the same length as the raw trajectory. Terminal frames are forward-filled from the last available logical value when the source logical action array is shorter than the raw frame sequence.

Reading the Data

Minimal Python example:

import json
import h5py

path = "datasets/libero_10/KITCHEN_SCENE3_turn_on_the_stove_and_put_the_moka_pot_on_it_demo.hdf5"

with h5py.File(path, "r") as f:
    demo = f["data/demo_0"]
    frame = 10

    agentview = demo["obs/agentview_rgb"][frame]
    eye_in_hand = demo["obs/eye_in_hand_rgb"][frame]
    robot_action = demo["actions"][frame]
    logical_action = demo["logical_actions"][frame].decode("utf-8")
    logical_state = json.loads(demo["logical_states"][frame].decode("utf-8"))

    print(logical_action)
    print(logical_state)
    print(robot_action)
    print(agentview.shape, eye_in_hand.shape)

Visualization App

The repository includes a browser-based HDF5 reader with no web framework dependency. It uses Python's standard HTTP server plus h5py, numpy, and Pillow.

[image]

Install dependencies:

python3 -m pip install -r requirements.txt

Run the viewer:

python3 visualize_dataset.py --host 127.0.0.1 --port 8000

By default, the viewer looks for datasets/ next to visualize_dataset.py, so it still works if the repository folder is renamed. Use --datasets-dir /path/to/datasets only when the data is stored elsewhere.

Path note: keep visualize_dataset.py and datasets/ in the same repository root when possible. If you move the HDF5 files, pass the new dataset directory explicitly:

python3 visualize_dataset.py --datasets-dir /absolute/path/to/datasets --host 127.0.0.1 --port 8000

Open:

http://127.0.0.1:8000

The app supports suite, task, demo, and frame selection, playback controls, both RGB camera streams, logical action, formatted logical predicates, and robot action/state values.

Data Processing Notes

The logical labels were merged from files named like:

synced_final/{suite}_{task_id}_{demo_id}_pairs.hdf5

into the corresponding HDF5 files under datasets/{suite}/. The merge script validates task-file correspondence using RGB frame equality before writing logical fields.

The current dataset has already been merged and filtered. Demos without logical annotations were removed, remaining demos were renumbered contiguously, and num_demos / total attributes were updated.

Intended Use

This dataset is intended for research on robot manipulation, imitation learning, planning-conditioned policies, language/logical-state grounding, and analysis of action/state abstractions in LIBERO demonstrations.

Limitations

• —The logical annotations are aligned to existing rendered/simulator frames and should be treated as derived labels.
• —HDF5 files are not automatically previewable in the Hugging Face dataset viewer like tabular formats.
• —Users should verify compatibility with their downstream LIBERO or imitation-learning pipeline, especially if the pipeline assumes exactly 50 demos per task.

Citation

If you use this dataset, please cite both the H-WM work associated with these logical annotations and the original LIBERO benchmark:

@article{huang2026hwm,
    title   = {H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Model},
    author  = {Huang, Jinbang and Chen, Wenyuan and Li, Zhiyuan and Pang, Oscar and Hu, Xiao and
               Zhang, Lingfeng and Hu, Yuanzhao and Zhang, Zhanguang and Coates, Mark and
               Cao, Tongtong and Quan, Xingyue and Zhang, Yingxue},
    journal = {arXiv preprint arXiv:2602.11291},
    year    = {2026}
}

@article{liu2023libero,
    title   = {LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning},
    author  = {Liu, Bo and Zhu, Yifeng and Gao, Chongkai and Feng, Yihao and Liu, Qiang and
               Zhu, Yuke and Stone, Peter},
    journal = {arXiv preprint arXiv:2306.03310},
    year    = {2023}
}