ww-249/pick-and-place-the-banana-for-franka-research3

Pick and Place the Banana for Franka Research 3

This repository contains Franka Research 3 robot data for the task:

pick up the banana and place it into the plate

The data was converted from local ROS bag recordings into OpenVLA-style intermediate episodes, no-op-trimmed intermediate episodes, and RLDS / TFDS training data.

Repository Contents

intermediate/
intermediate_no_noops_hf/
rlds_no_noops/
manifests/
rlds_dataset_builder/
OPENVLA_FRANKA3_DATASET.md
README.md

intermediate/

Original, untrimmed OpenVLA-OFT intermediate data.

Each episode directory contains:

data.jsonl
metadata.json
images/*.jpg

Use this directory when you need the original converted episodes before no-op trimming, for inspection, debugging, or rebuilding derived datasets.

intermediate_no_noops_hf/

No-op-trimmed OpenVLA-OFT intermediate data prepared for upload to Hugging Face.

It has the same episode-level structure as intermediate/, but idle / no-op segments were removed. Use this directory when you want the trimmed intermediate representation rather than the prebuilt RLDS files.

rlds_no_noops/

No-op-trimmed RLDS / TFDS build output.

This is the preferred directory for OpenVLA-style training pipelines that read RLDS / TFDS datasets directly. It contains the built TFRecord shards and TFDS metadata for:

franka3_banana_plate_easy_success
franka3_banana_plate_hard_success
franka3_banana_plate_non_holdout_success

manifests/

Split metadata for the original success episodes:

split_manifest.csv
split_summary.json

These files document which episode belongs to each split.

rlds_dataset_builder/

TensorFlow Datasets builder code used to convert the OpenVLA intermediate episodes into RLDS / TFDS datasets.

The builders define:

• —dataset names
• —train / test split layout
• —observation, action, proprioception, and language fields
• —image loading
• —episode and step serialization into RLDS format

The code is useful if you need to rebuild RLDS data from the intermediate episodes or inspect the exact schema used by the TFDS datasets.

Splits

Only successful episodes are used.

Episodes are sorted by timestamp / episode directory name:

• —first 150 success episodes: easy_success/train
• —next 10 success episodes: easy_success/test
• —remaining 47 success episodes: hard_success/train

Summary:

Important: easy_success/test is a held-out test set and should not be mixed into training.

Dataset Variants

There is no intermediate_no_noops/ directory in this Hugging Face repository. That name was used for a local working directory and was intentionally not uploaded.

Data Fields

The RLDS datasets contain the following step-level fields:

• —observation/image: RGB image, 224 x 224 x 3
• —observation/state: proprioceptive state, 15 dimensions
• —action: robot action, 7 dimensions
• —language_instruction: task instruction string
• —is_first, is_last, is_terminal: RLDS episode markers

Current conventions:

• —action coordinate frame: franka_base
• —action dimension: 7
• —proprioception dimension: 15
• —intended action chunk length for OpenVLA-OFT training: 16

Recommended Usage

Use rlds_no_noops/ if your training code reads RLDS / TFDS directly.

Use intermediate_no_noops_hf/ if you need to rebuild RLDS or inspect the trimmed episode data.

Use intermediate/ only when you need the original untrimmed converted episodes.

See OPENVLA_FRANKA3_DATASET.md for more detailed notes about the conversion, splits, builder code, and training environment.