galbenecho/echo-foundry-vla
Echo Foundry — VLA Dataset TL;DR 10,133 episodes (60 h) of physics-verified robot manipulation where every tick explains itself — a reasoning sentence with live measured distances on all 4.3M frames, every object's ground-truth pose, metric depth, typed outcomes with the evidence attached, and the planner's rejected alternatives with the margins that killed them. One robot (UR5e), five tabletop tasks, simulation registered to a real robot cell whose calibration… See the full description on the dataset page: https://huggingface.co/datasets/galbenecho/echo-foundry-vla.
Echo Foundry — VLA Dataset
TL;DR
10,133 episodes (60 h) of physics-verified robot manipulation where every tick explains itself — a reasoning sentence with live measured distances on all 4.3M frames, every object's ground-truth pose, metric depth, typed outcomes with the evidence attached, and the planner's rejected alternatives with the margins that killed them. One robot (UR5e), five tabletop tasks, simulation registered to a real robot cell whose calibration ships inside.
from lerobot.datasets.lerobot_dataset import LeRobotDataset
ds = LeRobotDataset("galbenecho/echo-foundry-vla", episodes=[0, 1, 2]) # subset: seconds
ds[0]["extra.language.decision"]
# "Lowering the white flask onto the green cube's top center, 215 mm to go."What you actually get
- 60 hours of manipulation where every frame explains itself. Not captions — computed sentences: "Lowering the white flask onto the green cube's top center, 215 mm to go." The 215 is measured from the same row's state; you can recompute it.
- The failures, kept and diagnosed. 835 failed episodes with the physical reason and the millimetres. 1,057 flagged successes where the flag is a typed object carrying its proof (which gate, which contact, how deep, which frame).
- What the planner didn't do. ~4.4 rejected grasp candidates per episode, each with the reason and the margin that killed it. No collected dataset has this channel — a human demonstrator can't log what they considered.
- The world itself. Every object's ground-truth pose every 50 ms, metric depth per camera, and each episode's initial world as a loadable MuJoCo model.
- A real cell behind it. The simulation is registered to a physical robot cell; its calibration, measured depth noise, and actual room ship in
calibration/.
Start here → [USAGE.md](USAGE.md) — seven copy-paste recipes and the map of what links to what. Ten minutes to productive.
Why this exists
A human demonstrator can't log what they considered and didn't do. A solver can — and this corpus was made by one that searched, gated, and measured everything it touched. Mean 4.4 rejected grasp candidates per episode, each with the physical reason and the margin in mm.
What's in the box
The five tasks
Each scene contains ONLY what its row lists — there is no scene with four cubes and a flask. Structures below are read from the shipped placement records (all 132 distinct sequences across 10,133 episodes enumerated — nothing else occurs).
Outcomes: three tiers, evidence attached
A success tier requires achievement evidence — recorded, seated placements. next.success=True only on clean_success.
outcome_flags are typed objects carrying their proof (severity 3→1):
[{"kind": "gate_refused", "severity": 3,
"search_refusals": {"flight_margin": 13, "arm_collision": 8, "workspace": 1},
"worst_margin_mm": {"flight_margin": 10.8, "workspace": 70.0}},
{"kind": "unplanned_contact", "severity": 2,
"contacts": [{"a": "work_table", "b": "ur5e_robotiq", "depth_mm": 0.4,
"frame": 142, "phase": "in_transit"}]},
{"kind": "bystander_disturbed", "severity": 1,
"disturbed": [{"slug": "cube_3", "moved_mm": 29.8}]}]<details> <summary><b>Gate glossary — what each margin number measures</b></summary>
</details>
Sim-to-real
Built for the zero-shot experiment: train here, evaluate on the physical cell this data is registered to — its calibration, depth noise, and room ship in calibration/. Real-trial measures are the same columns the synthetic verdicts carry, so sim and real compare column-for-column. Real-robot results not yet included; when published they'll use this protocol.
<details> <summary><b>Format details (depth codec · worlds · splits · loading)</b></summary>
Layout — LeRobot v3.0, chunked ≤1000 files/dir. Full download ~117 GB; use episodes=[…] for subsets.
Depth — depth/observation.depth.<cam>/chunk-CCC/file-EEE.u16seq: uint16 millimetres, LZMA. Frame count = episode length + 1 (terminal observation). Foreground = render ground truth; background = the real room's measured depth (dropouts = 0, ~79% valid). Reader (stdlib only):
import lzma, numpy as np
b = open(path, 'rb').read()
nl = b.index(b'\n'); n, h, w = (int(v) for v in b[:nl].split())
depth = np.frombuffer(lzma.decompress(b[nl+1:]), dtype=np.uint16).reshape(n, h, w)Worlds — worlds/models/<family_id>.xml: the episode's initial world as a standalone MuJoCo model (shared meshes/textures in worlds/assets/). extra.kernel.initial_state_sha1 is the identity key of the authored source world — episodes sharing it started byte-identical; source-level verification available on request.
Splits — info.json declares train/val cut at a counterfactual-family boundary: sibling episodes share a byte-identical initial world; a row-level split would leak starts across sides.
Companion — `echo-foundry-world` regroups a subset into counterfactual families (same start, different action order, divergent outcomes). The sets overlap — don't train on one and evaluate on the other.
</details>
Provenance
Authored worlds → solved plans → physics rollouts → rendered film, composited over the real cell's captured room. Full corpus 100.7 authored hours at a measured $0.49 per motion-hour on one RTX 3090 — scaling is a compute purchase, not a collection program.
Contact: Echo Labs — sapir@echo-robots.com