ytung/g1-pipette-augmented

G1 pipette: 544 validated simulation augmentations

544 independently validated simulated demonstrations of attaching a green pipette tip from a rack and lifting it. These are the new augmentation campaigns only; the earlier 600 demonstrations and real videos are not included.

309 placement-only variants and 235 joint-approach variants. Joint variants use smooth 1°/3° command offsets, return toward the baseline before contact, and may include fitted rack/tip placement. Every accepted capture passed independent simulation replay. These are correlated descendants of 12 parents, not 544 independent real-world task contexts.

Ego-camera correction — release v2

The ego videos for 513 episodes were re-rendered using the intended dataset/session camera settings. The remaining 31 episodes already used that pose. Both LeRobot formats and browser previews include the correction. Episode IDs, parent splits, dense commands, numeric trajectories, independent physics success results, and side/right-wrist camera videos are preserved. The render revision restores full saved simulator states; it does not generate new demonstrations or claim a new physics rollout. Camera fits remain approximate.

Use a fresh download directory (or verify the updated checksums) when replacing a download of the earlier release. The v3.0 tag now selects this corrected release; previous commits remain available for reproducibility.

Watch the demonstrations

Open the episode video viewer. Select train or validation; each row is one complete demonstration. Videos show fixed ego (left), fixed side (center), and right wrist (right) together. original_episode identifies the real parent; augmentation identifies placement randomization or joint perturbation. episode_index matches the canonical v3 dataset.

<video controls preload="metadata" src="https://huggingface.co/datasets/ytung/g1-pipette-augmented/resolve/main/previews/episode_000000.mp4" width="100%"></video>

The browser previews are H.264/yuv420p viewing copies. The numerical labels and non-ego training videos remain unchanged; ego training videos use the camera correction described above. previews/ and viewer/ are for browsing, not training inputs. The native MP4s use MPEG-4 Part 2; many browsers cannot play that codec. The former 0 observation.images.rgb label was the Hub's automatic camera-folder classification, not an episode ID or a missing image.

Download on another computer

This repository is public. Authentication is optional for download; do not place tokens in scripts.

python -m pip install -U huggingface_hub
hf auth login
hf download ytung/g1-pipette-augmented --repo-type dataset \
  --local-dir ./g1-pipette-augmented
python ./g1-pipette-augmented/verify_download.py ./g1-pipette-augmented

Use --revision <commit-sha> to pin an exact dataset release. The repository commit is available on the Hub. No simulator, original workstation paths, or original real-world dataset is required for loading/training.

Layout and split handling

• —meta/, data/, videos/: standard LeRobot v3.0, all 544 episodes. Training IDs 0–466; validation IDs 467–543. meta/info.json and meta/parent-splits.json identify membership.
• —compat/v21/sim_train/: LeRobot v2.1, 467 episodes, reindexed from zero.
• —compat/v21/sim_val/: LeRobot v2.1, 77 episodes, reindexed from zero.
• —training/: StarVLA Qwen-OFT example, custom modality registry, launcher, and loader smoke test.
• —checksums.json: SHA-256 manifest of release files, excluding itself and this verifier's runtime files.

The canonical meta/stats.json uses training episodes only for normalization (including images). Per-episode metadata still describes every episode.

LeRobot loaders do not necessarily select the train span automatically. For v3, explicitly pass episodes=list(range(467)) for training; use list(range(467,544)) for validation. Do not randomly split frames or descendants of the same real parent. When mixing real data or the earlier 600 simulations, preserve the existing parent-level split across every source.

from lerobot.datasets.lerobot_dataset import LeRobotDataset
train = LeRobotDataset(
    'ytung/g1-pipette-augmented', root='./g1-pipette-augmented',
    episodes=list(range(467)), video_backend='pyav',
    delta_timestamps={'action': [i / 30 for i in range(16)]},
)

This example returns 29-dimensional motor actions. The separate hand columns are preserved in the dataset. The supplied StarVLA adapter concatenates them to 41 dimensions.

Observation and action contract

30 Hz; absolute joint targets, not joint deltas, end-effector poses, or the original 35-D high-level action interface.

Exact joint order is in meta/info.json. Hand register order is pinky, ring, middle, index, thumb bend, thumb rotation. The hand broke during original data collection and fingers were taped around the pipette. Preserved hand-command labels may include reconstructed/imputed values; they are not evidence of working adaptive finger control. The saved hand state is simulated, not measured real hand feedback.

The 30 Hz actions sample executed dense commands. They do not guarantee identical physics when replayed with 30 Hz zero-order hold. Camera calibration, fixed pelvis, pipette/hand mechanics, and the geometric attachment constraint limit simulation fidelity. Simulation success does not establish real-world transfer.

Fine-tuning Qwen-OFT with StarVLA

Install StarVLA in a CUDA environment following its installation and training guide. The supplied recipe targets its QwenOFT framework, action_horizon API, and auto-discovered examples/**/train_files/data_registry interface. See environment.json for the locally tested source revision and package versions. A full model training run has not been performed as part of dataset preparation.

# Use absolute paths for these two variables.
export STARVLA_ROOT=/path/to/starVLA
export DATASET_ROOT=/path/to/g1-pipette-augmented

# In your StarVLA Python environment:
python "$DATASET_ROOT/training/smoke_dataset.py" \
  --starvla-root "$STARVLA_ROOT" \
  --data-root "$DATASET_ROOT/compat/v21"

# Copies this dataset's uniquely named registry into your StarVLA checkout.
bash "$DATASET_ROOT/training/train.sh"

The example trains on sim_train only, with 3 cameras, a 41-dimensional action head, 16 future steps, physical 0–1000 hand normalization, and no proprioceptive input. It starts from Qwen/Qwen3-VL-2B-Instruct; the robot action head must learn this action schema. A pretrained 7-D or 30-D robot action head is not directly compatible. Batch size defaults to 2; use BATCH_SIZE, NUM_GPUS, and BASE_VLM to configure the destination hardware. Memory requirements depend on model, optimizer, and GPU; the example is not a guarantee of fitting a particular card.

Keep sim_val out of the training mixture. For actual evaluation, use training/checkpoint normalization statistics on validation data; do not normalize each evaluation set independently. The supplied loader smoke test checks schema and decoding, not policy quality. Evaluate held-out real task success separately when assessing transfer.

The dataset also supports other VLAs with an explicitly matching state/action adapter. The 29-D main action and optional 12 hand registers are distinct from previous high-level or piston-control schemas.

Validation and provenance

meta/dataset-audit.json records checks of all exported numeric rows, executed command interpolation, every episode's first/last frames across all cameras, and future-action padding. starvla-loader-audit.json records actual StarVLA sample loading and 41-D action assembly. Portable episode-to-parent/family mappings are in meta/parent-splits.json; meta/augmentation.jsonl retains capture lineage and hashes. Local source paths in provenance are historical references, not loading dependencies.

Failed attempts, raw real videos, and simulator replay archives remain on the collection workstation and are intentionally outside this training release. Original collection data is unchanged.