shibuina/drawvla-prompt-validation

DrawVLA — Sketch-Prompt Validation

Circle (which) + arrow (where) + caption (what) visual instructions overlaid on LIBERO observations, each labelled with a binary verdict for training a prompt validator or a self-checking VLA:

• —`right` — every channel is correct and exactly one reading survives; execute.
• —`wrong` — a channel is incorrect or the deictic prompt remains under-determined; reject. Formerly ambiguous prompts are retained in this class.

All captions are name-free L2/L3 deixis (for example, "move this there"). Object identity must come from the circle/arrow sketch, not an explicit noun. n_readings = n_referents x n_destinations is retained as diagnostic metadata. Embedded images use JPEG encoding.

Train and validation rows include label_reason, a human-readable explanation derived from the channel checks and declared grounding count. The demo-disjoint test split keeps its ground-truth verdict for scoring but sets reason_available=false and nulls all reason/rationale fields (label_reason, note, mode/category, failure-channel flags, severity, and resolution diagnostics), preventing rationale leakage during final testing.

Generated from the failure taxonomy (failure_modes.md). Split into 7 configs by family; each loads independently:

from datasets import load_dataset
ds = load_dataset("shibuina/drawvla-prompt-validation", "crossmodal", split="train")

Configs (37,988 samples)

Fields

image, id, mode, category, tier, solution, verdict, right, reasonavailable, labelreason, circleok/arrowok/captionok/consistencyok/sufficientok, severity, caption, captionlevel, nreferents/ndestinations/nreadings, resolvedby, groundingkind, renderednreadings/renderverified, circlejson, arrowjson, nameexplicit, note. The sketch geometry (`circlejson, arrow_json`, image-plane, normalized) lets you re-render at any resolution. This release samples only L2 (relational deixis) and L3 (pure deixis); L0/L1 are excluded because they explicitly name the task or object category. Splits are approximately 80/10/10 by demo; no demo straddles train, validation, or test.

Built with the pipeline in data_gen/ on the OpenVLA modified_libero_rlds demos.