Datasetpublic

lithyeon/franka_pnp_big100_base

Franka PnP Big-100 — Base 100 shelf pick-and-place demonstrations with no clutter: one coke can on the floor of a large shelf, a terracotta dish on the desk to the left. Collected in Isaac Lab (Isaac Lab Arena) with a Franka Panda and a cuRobo-planned scripted expert. One successful episode per scene; the can position is sampled uniformly over the shelf floor (see Sampling). This is one of a pair of datasets: franka_pnp_big100_base (no clutter) and franka_pnp_big100_distract… See the full description on the dataset page: https://huggingface.co/datasets/lithyeon/franka_pnp_big100_base.

sourceHugging Faceupdated 12d agoView on Hugging Face

0likes222downloads

Dataset Card

Franka PnP Big-100 — Base

100 shelf pick-and-place demonstrations with no clutter: one coke can on the floor of a large shelf, a terracotta dish on the desk to the left.

Collected in Isaac Lab (Isaac Lab Arena) with a Franka Panda and a cuRobo-planned scripted expert. One successful episode per scene; the can position is sampled uniformly over the shelf floor (see Sampling). This is one of a pair of datasets: franka_pnp_big100_base (no clutter) and franka_pnp_big100_distract (paired, non-blocking clutter).


Episodes / frames	100 / 35903
Dataset fps	10 (recorded at 50 Hz, every 5th step)
Episode length	mean 35.9 s (min 19.4 s, max 48.9 s)
Cameras	3 × 320×320 RGB, h264
Robot	Franka Panda (Franka Hand), base at (−0.45, 0, 0.72) m
Format	LeRobot v2.1 (`meta/info.json`), GR00T `meta/modality.json`
Task	"Pick up the coke can and place it in the terracotta dish."

Scene

—Desk 0.76 × 1.70 m, top at z = 0.76 m. Big shelf on the desk: interior 0.45 (depth) × 0.95 (width) × 0.60 m (height), mouth facing the robot, walls 3 cm.
—Target: Coke can (Objaverse 013b0fff…, Ø 6.6 cm). Receptacle: terracotta dish at (−0.20, −0.70) m on the desk, left of the shelf, fixed.
—Success = the can rests inside the dish at the end of the episode.

Sampling

—Can position: uniform over the shelf floor, keeping 10 cm (the gripper half-width) from the side/back walls and from the mouth plane: x ∈ [−0.065, 0.185], y ∈ [−0.375, 0.375] m (robot frame; +x into the shelf). 100 accepted positions span x −0.064…0.184, y −0.358…0.361.
—Acceptance: a position is kept if the expert succeeds within its first 40 planned grasp candidates; 28 sampled positions were discarded (no success in the batch) and 4 more were dropped because no distractor pairing succeeded. This biases the set toward positions the scripted expert can solve; the left/right shelf edges are under-represented (front/back × left/center/right: {"back-center": 21, "back-left": 17, "back-right": 11, "front-center": 16, "front-left": 23, "front-right": 12}).

Expert

—Grasps: 1000 MolmoSpaces-style annotated grasps on the can; grasp + 1 cm pre-grasp IK-checked with cuRobo (shelf/desk collision, joint-limit margin 0.12 rad); ranked by MolmoSpaces cost + axis-offset penalty, near-frontal grasps (|approach yaw| ≤ 30°) first.
—Plan: cuRobo trajectory to the pre-grasp, 1 cm straight advance, 3 cm lift, 1 cm straight extract, retrieval to above the dish, release.
—Execution: absolute joint-position targets at 50 Hz, linear interpolation along the planned dense path at 0.6 rad/s (0.3 rad/s while carrying), settle to 0.005 rad at segment ends, Gaussian target noise σ = 0.002 rad (0.11°) per step. Ramp targets anchored on the previous waypoint (fix of 2026-09-08; earlier data had a saw-tooth wrist jitter).
—Per-attempt success rate of the expert ≈ 10–16 %; episodes here are the first success per scene (mean 4.5 failed attempts before it).

Features

key	shape	meaning
`observation.state`	8	pandajoint1–7 (rad) + pandafinger_joint1 (m, 0.04 open)
`action`	8	absolute joint targets panda_joint1–7 (rad) + gripper command (+1 open / −1 close)
`observation.images.exterior_image_1_left`	320×320×3	left tripod camera at (−0.55, −0.45, 1.30) m looking at (0.05, −0.22, 0.81) m
`observation.images.exterior_image_2_left`	320×320×3	right tripod camera (y-mirror of the left one)
`observation.images.wrist_image_left`	320×320×3	wrist camera on the hand's −x face, looking along the fingers

Cameras use the ZED Mini lens (f = 2.8 mm, 5.376 mm aperture) at a square 320×320 sensor. Camera keys follow DROID naming.

Extra files

—meta/scenes.jsonl — one row per episode: scene name, seed, can xy, dish xy, executed grasp (candidate rank, annotation index, approach yaw), failed attempts, raw frame count, paths to the scene files.
—meta/collection.json — expert and camera settings.
—scenes/<scene>/layout.yaml — full scene description (desk, shelf boxes, target, receptacle, obstacles by asset uid); scenes/<scene>/plan_cNNN.yaml — the executed grasp chain (joint paths per segment).
—assets/<uid>/ — visual OBJ/MTL/PNG, collision OBJ and rigid USD for the can, the dish and every distractor referenced by the layouts (Objaverse-derived; per-object authors and licenses in ATTRIBUTION.md: 99 × CC BY, 1 × CC BY-SA, 2 × CC BY-NC).
—eval/ — 20 held-out evaluation layouts sampled the same way (no filtering by expert success). Each layout.yaml carries init_arm_q, the episode-start arm pose every demo was recorded from ([0, -1.3, 0, -2.5, 0, 1.5, 0.8] rad, gripper open 0.04 m); start evaluation episodes from it so the first observations match the training distribution (also in meta/collection.json as episode_start_arm_q).

Pairing

Episode i of franka_pnp_big100_base and episode i of franka_pnp_big100_distract are the same scene (same can position); see PAIRING.md.

Known limitations

—Scripted expert with ≈ 10 % per-attempt success: every episode is a success, but the set of positions is biased toward what the expert could solve (see Sampling).
—Distractors never sit close to the target (≥ 35 cm) by construction.
—Simulation only (PhysX); the can is a rigid convex-decomposed mesh.
—Receptacle is always on the left.