ccwatson/directional_pick_place_simpledesk_diverse_targets_tested40_200demos_20260716_molmobot

Directional SimpleDesk Pick-and-Place Diverse Targets

This dataset contains 200 successful scripted Franka demonstrations converted from native MolmoSpaces output into the MolmoBot/Synthmanip training layout. The task is to pick up one tabletop object and place it either to the left of or to the right of a second object, from the robot's point of view.

The dataset is balanced by direction: 100 demonstrations use left prompts and 100 use right prompts. It covers 40 fixed initial arrangements (20 per direction), with five successful demonstrations from each arrangement. Pickup and reference roles vary independently; the bowl is not a fixed target.

This is the directional dataset. It does not contain demonstrations from the paired, non-directional vanilla place next to control benchmark.

Dataset contents

Each arrangement is stored as one house_* directory. Each directory contains five HDF5 files with one traj_0 trajectory apiece and ten MP4 files (one video per camera per trajectory). The two camera streams are:

• —wrist_camera_zed_mini
• —droid_shoulder_light_randomization

The HDF5 files retain joint and end-effector actions, robot observations, camera calibration, object and robot state, strict and lax success signals, and the rich MuJoCo state needed for replay or rerendering. Video paths inside the HDF5 files are relative to their containing house_* directory.

The split is arrangement-safe: benchmark episodes are sorted by episode ID, the last four (10%) are held out for validation, and all five attempts from an arrangement stay in the same split. valid_trajectory_index.json in each split is the trajectory index consumed by SynthmanipDataset.

Task and success semantics

Natural-language instructions have forms such as:

Pick up the wooden cup and place it to the left of the blue tissue box
Pick up the red mug and place it to the right of the brown egg

Strict task success requires all of the following:

1. 1.The pickup object's axis-aligned XY surface gap from the reference object is between 0 and 0.06 m. The configured maximum is 0.05 m and the task applies a 0.01 m numerical tolerance.
2. 2.Both objects are supported by the same surface.
3. 3.The reference object has moved no more than 0.15 m and rotated no more than 60 degrees from its initial pose.
4. 4.The pickup object lies inside the requested left or right 90-degree angular slice (plus or minus 45 degrees around the ideal direction), measured in the robot-relative tabletop frame.

lax_success is also recorded at every step. It applies conditions 1--3 but does not require condition 4. Therefore, a rollout may have lax next-to success without satisfying the directional instruction. The 200 retained trajectories all end in strict success.

Diversity and collection

The 40 benchmark arrangements use five pickup categories:

Cup (11), Mug (10), TissueBox (7), Potato (6), SprayBottle (6)

Reference-object roles span 15 categories:

Bowl, Box, Calculator, Candle, Cup, DishSponge, Egg, Mug, PaperTowel,
Potato, RemoteControl, SoapBottle, SprayBottle, TissueBox, Tomato

Each scene contains the pickup object, the reference object, and two additional tabletop objects. All arrangements use the same SimpleDesk house (1088) but vary object identities, roles, and poses.

The source benchmark was screened with the pick_and_place_relative_next_to_search scripted policy. Every retained arrangement achieved at least one strict success in five screening attempts. Dataset collection then requested five strict successes per arrangement and finished with 200 successes from 210 rollout attempts using a 500-step horizon.

MolmoBot layout and normalization

This repository is ready for the MolmoBot SynthmanipDataset loader:

dataset_manifest.json
directional_pick_place_simpledesk_diverse_targets_tested40_200demos_20260716_norm_stats.yaml
train/
  house_*/
  valid_trajectory_index.json
val/
  house_*/
  valid_trajectory_index.json

Normalization statistics were computed from the training split with the franka_joint action preset and franka_droid camera preset. Actions and robot state are both 8-dimensional. Actions use quantile normalization; state uses min/max normalization.

The post-conversion loader smoke test produced:

The four images are two observation frames from each of the two cameras. See dataset_manifest.json for the exact split membership, copied-file inventory, validation counts, normalization settings, and smoke-test result.

Download

hf download \
  ccwatson/directional_pick_place_simpledesk_diverse_targets_tested40_200demos_20260716_molmobot \
  --repo-type dataset \
  --local-dir directional_pick_place_simpledesk_diverse_targets_tested40_200demos_20260716_molmobot

Point MolmoBot training at the downloaded root and use the normalization YAML at that root. The franka_joint action and franka_droid camera presets used during conversion should also be used when constructing the training dataset.

Validation

The native MolmoSpaces validation pass found zero invalid trajectories. During conversion, all 400 referenced camera videos were resolved and copied with zero missing files. Both train and validation splits passed a real SynthmanipDataset item-load smoke test. The published payload contains 200 HDF5 files and 400 MP4 files.

Provenance and reproduction

Source benchmark, relative to the MolmoSpaces repository:

benchmarks/custom/molmospaces-bench-v2/procthor-objaverse/
FrankaPickAndPlaceRelativeNextToSimpleDeskDiverseTargetsTested40Bench/
FrankaPickAndPlaceRelativeNextToSimpleDeskDiverseTargetsTested40Bench_20260716_json_benchmark

Provenance hashes:

benchmark.json SHA-256: 23511589a1c58c7d3ce27d841a93900ac502a71c68daeca91682ffc6a44e12b9
complete benchmark-leaf SHA-256: 83544065ae11b2c3f46f7463614c6052676621e4334bbb9bc8639d35647fc199
MolmoSpaces collection commit: 6b752b553a4f2c751c2fb0fa6f847916988e96af
initial Hugging Face data revision: 5446191605f7eb3020983d83f707153d9719ecfb

The collection manifest records that the MolmoSpaces worktree contained uncommitted benchmark-related changes at collection time. The two benchmark hashes above bind this dataset to the exact screened artifact independently of that worktree state.

The MolmoBot conversion and upload were run from the coordination stack's components/MolmoBot directory:

STACK_ROOT=/home/christopher/Documents/molmo-robotics-stack
SOURCE_NAME=directional_pick_place_simpledesk_diverse_targets_tested40_200demos_20260716
SOURCE="$STACK_ROOT/scratch/local_data/datasets/$SOURCE_NAME"
DEST="$STACK_ROOT/scratch/local_data/datasets/${SOURCE_NAME}_molmobot"
REPO="ccwatson/${SOURCE_NAME}_molmobot"

cd "$STACK_ROOT/components/MolmoBot"
PYTHONPATH=MolmoBot PYTHONUNBUFFERED=1 \
  MolmoBot/.venv/bin/python \
  MolmoBot/scripts/data/convert_molmospaces_to_molmobot.py \
  --source-root "$SOURCE" \
  --dest-root "$DEST" \
  --molmobot-root "$STACK_ROOT/components/MolmoBot" \
  --repo-id "$REPO" \
  --upload

The original native collection command and its full options are preserved in the source run's metadata/run_manifest.json and in DIRECTIONAL_PICK_PLACE_SIMPLEDESK.md in the MolmoSpaces repository.

Limitations and asset terms

• —This is a small, scripted-policy dataset from 40 fixed arrangements in one SimpleDesk house, not a broad measure of household-scene generalization.
• —Five demonstrations share each initial arrangement, so trajectory count must not be mistaken for 200 independent layouts.
• —The data contains only successful scripted rollouts and therefore does not represent the full failure distribution of the task or planner.
• —Object coverage is diverse but uneven, and behavior reflects the scripted planner's grasp and motion biases.
• —Only left and right are included; in front of and behind are absent.

No standalone license is asserted by this dataset card. The scenes depend on MolmoSpaces, ProcTHOR, Objaverse, and their associated assets. Users are responsible for following the software and asset terms that apply to their use.