stray-light/zerograv-manipulation-trajectories-v0

## ⚠️ Superseded — do not use for distillation These trajectories were collected from policies trained before the sim-real action-space alignment. Measured on this data: mean |arm action| is 0.72, with 46-53% of actions at the clip boundary of [-1, 1]. For comparison, the real robot's own pi0.5-DROID policy outputs a mean of 0.066 and human teleop 0.101 — so these actions sit 7-11x outside the distribution the base model has seen. Distilling them would push pi0.5-DROID's action head far outside its training distribution. They also predate several sim-real alignment fixes: real gripper timing (0.9 s to close, vs 0.1 s here), torque rate limiting, gravity-residual bias torque, a 40 N contact reflex, and home-pose noise (the arm starts from an identical pose in every episode here — the first 13 state dims have zero variance). A replacement collected with arm_action_bound=0.2 is in progress. Kept public for provenance and reproducibility only.

Zero-Gravity Manipulation Trajectories (DROID rig)

Expert trajectories from PPO policies trained under zero-gravity manipulation dynamics, on a hardware-matched simulation of a real Franka Panda + Robotiq 2F-85 rig ("DROID"), collected for VLA distillation research. Simulated in ManiSkill3 under the hybrid_impedance_delta control mode, which reproduces the real robot's low-level torque/friction dynamics (not an idealized PD controller) — see PickCube-DROID-* results below for why this matters.

Tasks

Each is that run's best_ckpt.pt (highest eval/success_once seen during training), rolled out deterministically. The first three were trained for 100M steps with num_envs=4096, seed 1; pusht-droid for 200M steps at ent_coef=0.002 (see the QA filtering section below — this task also needed a higher entropy coefficient and longer horizon to converge; ent_coef=0.001 on the same task and budget stayed near 0). The checkpoints themselves are not published here — they live with the training runs (W&B project Zerograv-Distillation), not on the Hub.

Each subfolder is an independent LeRobotDataset. observation.state is the flattened ManiSkill state vector for that task (dimensionality differs per task, so the subfolders can't share a single schema); observation.images.* has one video per camera, rendered with plain rasterization (not ray tracing; ManiSkill's RT path has confirmed rendering bugs for this robot/scene that plain rasterization doesn't have). Schema note: pusht-droid additionally carries episode.success_once/ episode.success_at_end boolean fields (added to the export pipeline after the first three tasks were collected); the other three subfolders don't have them — expect this if loading multiple subfolders with shared code.

Loading

This repo holds three datasets as subfolders, which LeRobot doesn't natively address (it assumes one repo == one dataset, and root= means "local directory holding the dataset", not "remote subfolder to fetch"). So download the subfolder first, then point root= at the local result:

from huggingface_hub import snapshot_download
from lerobot.datasets.lerobot_dataset import LeRobotDataset

REPO = "stray-light/zerograv-manipulation-trajectories-v0"
TASK = "pushcube-droid"   # or liftpeg-droid, pickcube-droid-longhorizon, pusht-droid

snapshot_download(repo_id=REPO, repo_type="dataset",
                  allow_patterns=[f"{TASK}/*"], local_dir="/tmp/zerograv")
ds = LeRobotDataset(REPO, root=f"/tmp/zerograv/{TASK}")

Passing root="pushcube-droid" directly does not work — it is read as a local path.

QA filtering

Episodes here are not simply every rollout the stock task called successful. Stock ManiSkill success for these tasks checks object position but not whether the object actually came to rest, so a cube sliding through the goal region at speed counts as a success. Filters run over the recorded episodes, but not the same set for every task — see the per-task notes below before assuming uniform methodology across subfolders:

• —Settle check (pushcube-droid, liftpeg-droid, pusht-droid): the primary object's linear velocity must drop below 0.05 m/s on at least one step where the stock success condition also holds. Rejects transient pass-throughs.
• —Ejection check (all four): rejects an episode if the object's velocity exceeds 1.0 m/s within the first 5 steps after reset — the signature of a PhysX depenetration artifact, where an object spawned overlapping geometry gets flung rather than manipulated.
• —Success-at-end check (pusht-droid only): additionally requires the stock success condition still hold on the episode's final recorded step, not merely at some point during it — a stricter, distinct metric from "succeeded at any point" (success_once). Not applied to pushcube-droid/liftpeg-droid/pickcube-droid-longhorizon, which predate this filter; those three are not guaranteed to hold success through to their last frame, only to have passed the settle/ejection checks above. Revisit those three if this inconsistency matters for your use case — re-collecting them under this filter is a known, not-yet-done follow-up.

pickcube-droid-longhorizon uses the ejection check only. It's a grasp-and-hold task, not release-and-settle, so absolute object velocity is the wrong yardstick for it (a held object under active control legitimately never settles) — the right filter for that task family is still an open question upstream, not yet decided.

What this cost, per task

Worth reading before you use these: `liftpeg-droid` retains only ~21% of its stock successes (29 of 141), the large majority lost to the ejection check — it is both the smallest and most aggressively filtered split here. pusht-droid has the highest yield of any task collected so far (220 of 256 attempts, 86%), reflecting its stronger checkpoint (success_at_end=0.727 in eval) rather than a looser filter — its filter is the strictest of the four (settle + ejection + success-at-end combined).

Episode lengths differ by design

pushcube-droid, liftpeg-droid, and pusht-droid were collected with a full-horizon setting — every episode runs to its full length even after success fires, because the object needs time after success for the settle check to mean anything. All three are therefore fixed-length per episode: 50 frames for pushcube-droid/liftpeg-droid, 100 frames for pusht-droid (that task's own max_episode_steps). pickcube-droid-longhorizon terminates on success as normal, so its episodes vary (~31.7 frames on average, 100-step limit). This is a collection-protocol difference, not a data defect.

Actions

action is 8-dimensional: 7 normalized arm joint deltas (scaled at 0.2 rad/unit, matching the real robot's controller) + 1 gripper command. These are the policy's raw output, not clipped to `[-1, 1]` — values outside that range occur and reflect what the network actually output, not what the simulated controller applied (the controller clips internally). Clip before using as direct training targets. A separately-maintained real-robot-format mirror of this same data does clip actions, matching what a physical robot would receive — see below.

Related repo

A sim-to-real-format mirror of these same episodes (for direct replay on physical hardware: clipped actions, joint-space only, no images) lives in `stray-light/zerograv-manipulation-realrobot-v0`. Same collection pass per task, so episodes correspond 1:1 with this repo (verified directly: 67/67, 29/29, 131/131, 220/220 across the four tasks).

License

CC BY 4.0. Purely synthetic data (simulated joint trajectories, actions, images, and rewards) — no personal or real-world sensor data.