Datasetpublic

justintiensmith/VLA_Benchmark_Prompted_1200

VLA Benchmark Prompted 1200 This is a non-destructive LeRobot v3.0 merge of six 200-episode benchmark datasets. It contains 1,200 training episodes and 609,553 frames. Every episode's task string is the assigned prompt from meta/benchmark/prompt_manifest_v3.csv. The Coke and Pepsi objects are canonically labeled as a crushed Coke can and a crushed Pepsi can. All rows are part of the training split. Active prompt design Prompt manifest v3 applies the exact 1… See the full description on the dataset page: https://huggingface.co/datasets/justintiensmith/VLA_Benchmark_Prompted_1200.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes92downloads

Dataset Card

VLA Benchmark Prompted 1200

This is a non-destructive LeRobot v3.0 merge of six 200-episode benchmark datasets. It contains 1,200 training episodes and 609,553 frames. Every episode's task string is the assigned prompt from meta/benchmark/prompt_manifest_v3.csv.

The Coke and Pepsi objects are canonically labeled as a crushed Coke can and a crushed Pepsi can. All rows are part of the training split.

Active prompt design

Prompt manifest v3 applies the exact 1,200 training prompts from VLA_Reasoning_Dataset_Info_Planned_After_61_Recollections.xlsx.

—All 1,200 prompts exactly match the supplied final manifest.
—State Recognition uses 49 explicit open/upside-down prompts and 151 category-only masked prompts.
—closed, upright, and ambiguous either language remain withheld from State Recognition training prompts.
—The other five task families retain the planned semantic and compositional masking rules from the supplied workbook.

The active machine-readable files are meta/benchmark/prompt_manifest_v3.csv and meta/benchmark/prompt_reassignment_audit_v3.csv.

State-recognition recollection overlay (40 episodes; historical)

On 2026-07-29, 40 checklist-selected episode slots were replaced, in checklist row order, with episodes 0–39 from `justintiensmith/MT_State_Recognition_40` at 33c4b378b5867aa82ef7fd0b4f6157f0c11f7a05. Existing episode prompts and task indices were preserved. The resulting dataset contains 614,367 frames.

State-recognition recollection overlay (61 episodes)

On 2026-07-30, 61 additional checklist-selected episode slots were replaced, in checklist row order, with episodes 0–60 from `justintiensmith/SP_State_Recognition_61` at 0a847ce8031c22a19e5adcbcc44d63b4d6cbd4dd. Together with the prior 40-episode overlay, 101 State Recognition demonstrations have now been recollected. The resulting dataset contains 609,553 frames and uses the v3 prompt manifest.

The source workbooks are retained under meta/benchmark/.

Immutable source revisions

—justintiensmith/MT_State_Recognition_200 at 168e12e735da9ad62716dca89cff660b53082cc3
—justintiensmith/SP_Relational_Placement_200 at 8eff46af18af7a9a6f763ed975f8b3357c673685
—justintiensmith/SP_Referential_Disambiguation_200 at 2642784f74a0c91bc118702b7993895aacd311f5
—justintiensmith/SP_Sequencing_200 at 1cda4c4c3c6ae1fa4b5dc6d2cf4c887bb9f1da4e
—justintiensmith/SP_Counting_200 at 6591c0f19952e23010b7c6a0630e4d60f8714cbb
—justintiensmith/MT_Size_Recognition_200 at 3a6e75d91e8339a87eebd8e545d87b148ae19925

Local identifier

justintiensmith/VLA_Benchmark_Prompted_1200

Episode 819 startup trim

Episode 819 (Put the blue pen in the bowl.) is unchanged after its initial 0.4 seconds. Twelve startup frames were removed to exclude one invalid wrist_flex.pos state reading at 0.2 seconds. The shared video payloads were not re-encoded; all five episode video start offsets advance by 0.4 seconds. The tabular data, indices, numeric episode statistics, and global statistics were rebuilt after the trim.