shibuina/drawvla-prompt-validation-clean

DrawVLA — Sketch-Prompt Validation

Circle (which) + arrow (where) + caption (what) visual instructions overlaid on LIBERO observations, each labelled with a binary verdict for training a prompt validator or a self-checking VLA:

• —`right` — every channel is correct and exactly one reading survives; execute.
• —`wrong` — a channel is incorrect or the deictic prompt remains under-determined; reject. Formerly ambiguous prompts are retained in this class.

All captions are name-free L2/L3 deixis (for example, "move this there"). Object identity must come from the circle/arrow sketch, not an explicit noun. n_readings = n_referents x n_destinations is retained as diagnostic metadata. Embedded images use JPEG encoding.

Train and validation rows include label_reason, a human-readable explanation derived from the channel checks and declared grounding count. The demo-disjoint test split keeps its ground-truth verdict for scoring but sets reason_available=false and nulls all reason/rationale fields (label_reason, note, mode/category, failure-channel flags, severity, and resolution diagnostics), preventing rationale leakage during final testing.

Generated from the failure taxonomy (failure_modes.md). Split into 6 configs by family; each loads independently:

from datasets import load_dataset
ds = load_dataset("shibuina/drawvla-prompt-validation-clean", "crossmodal", split="train")

Configs (28,720 samples)

Fields

image, id, mode, category, tier, solution, verdict, right, reasonavailable, labelreason, circleok/arrowok/captionok/consistencyok/sufficientok, severity, caption, captionlevel, nreferents/ndestinations/nreadings, resolvedby, groundingkind, renderednreadings/renderverified, circlejson, arrowjson, nameexplicit, note. The sketch geometry (`circlejson, arrow_json`, image-plane, normalized) lets you re-render at any resolution. This release samples only L2 (relational deixis) and L3 (pure deixis); L0/L1 are excluded because they explicitly name the task or object category. Splits are approximately 80/10/10 by demo; no demo straddles train, validation, or test.

Built with the pipeline in data_gen/ on the OpenVLA modified_libero_rlds demos.

Rendering variant

This is the fixed-style variant of shibuina/drawvla-prompt-validation: the same generator, seeds, corruption modes, captions and labels, but with every augmentation removed. Every prompt is rendered without the hand-drawn stroke model and without colour jitter — a perfect circle in one colour (RGB 230,40,30) and a straight arrow with a V head in one colour (RGB 30,90,230), both 3 px wide at 256 px — and every positive uses the exact ground-truth circle and arrow of its demonstration rather than a jittered placement, so the arrow tip and circle centre carry no augmentation either. Rows that became identical once the augmentation was removed (same declared circle, arrow and caption within a demo) were dropped, which is why this variant has 28,720 rows instead of 37,970 and a smaller positive share (11%). Row ids match the original dataset, so the two can be joined.