Datasetpublic

BiggerXu/rabench-vlabench-unified-libero-v1

VLABench Data Prep This directory contains an independent, non-Docker data conversion workflow for turning lerobot/libero into an episode-based HDF5 format that is easier for RABench agents to consume. Goal The source lerobot/libero dataset is distributed as: parquet tables for numeric columns mp4 video shards for image streams separate metadata parquet files for tasks and episode boundaries That structure is compact, but it is awkward for an agent to discover… See the full description on the dataset page: https://huggingface.co/datasets/BiggerXu/rabench-vlabench-unified-libero-v1.

sourceHugging Faceupdated 6mo agoView on Hugging Face

0likes6kdownloads

Dataset Card

VLABench Data Prep

This directory contains an independent, non-Docker data conversion workflow for turning lerobot/libero into an episode-based HDF5 format that is easier for RABench agents to consume.

Goal

The source lerobot/libero dataset is distributed as:

—parquet tables for numeric columns
—mp4 video shards for image streams
—separate metadata parquet files for tasks and episode boundaries

That structure is compact, but it is awkward for an agent to discover and use inside RABench. The workflow here converts it into one HDF5 file per episode with decoded frames and aligned metadata.

Output Layout

The converter writes:

—output/meta/tasks.json
—output/meta/dataset_info.json
—output/train/task_000/episode_000000.hdf5
—output/train/task_001/episode_000123.hdf5

Each HDF5 file follows schema.json.

Source Dataset Assumptions

Expected source root:

—/data/jiajunxu/RABench/vlabench_manipulation/vlabench_train/libero

Expected files:

—meta/info.json
—meta/tasks.parquet
—meta/episodes/chunk-000/file-000.parquet
—data/chunk-000/file-000.parquet
—videos/observation.images.image/chunk-000/file-000.mp4
—videos/observation.images.image2/chunk-000/file-000.mp4

Environment

This workflow is intended to run on the host machine, not in Docker.

Recommended conda env:

—data_process

Required Python packages:

—h5py
—datasets
—pyarrow
—opencv-python
—imageio
—numpy

Usage

Convert a small validation subset first:

bash

conda run -n data_process python convert_libero_to_unified.py \
  --source-root /data/jiajunxu/RABench/vlabench_manipulation/vlabench_train/libero \
  --output-root ./output \
  --max-episodes 5

Validate generated files:

bash

conda run -n data_process python validate_unified_dataset.py \
  --dataset-root ./output \
  --max-files 5

Notes

—The converted format preserves LIBERO's native 2-view setup. It does not try to fake the 4-camera VLABench runtime observation contract.
—Later, RABench can download the published converted dataset directly from your Hugging Face dataset repo during task preparation.