Ronaldo-GOAT/pose6daug

1# PORTABILITY — absolute paths baked into the copied code, and what they must become for ICLR2 3Nothing here was rewritten (copy-verbatim rule). This file lists what a thin4wrapper / env-var layer must supply before anything runs against the ICLR5workspace. It is the ICLR-specific delta on top of the two verbatim docs that6already enumerate every path in their trees:7 8- `docs/realvideo/PORTABILITY.md` — lineage A (stages 01-08 + drivers), grep-complete.9- `actaug/docs/PORTABILITY.md` — lineage B (actaug), grep-complete.10- neurips bundle `/lp-dev/jonghoon/___neurips_pose6daug___/migrator/PORTABILITY.md` — lineage C.11 12Counts below are occurrences in `*.py`/`*.sh` of this tree (excluding `actaug/` and `docs/`).13 14## 0. ICLR workspace layout the code must be pointed at15 16| ICLR location | contents | replaces (in the code) |17|---|---|---|18| `/lp-dev/jonghoon/iclr_pose6daug/data/<object>/<ep>/` with `recorded_videos/{37149196,11845904}_{left,right}.mp4`, `recordings/SVO/`, `teleoperation.h5`; objects `kanu`, `white_spray`, `green_pear`, `blue_cup`; plus `data/calibration/<ep>/` | raw episode captures (READ-ONLY) | `RL_EP_ROOT` / `EP_ROOT` = `/lp-dev/jonghoon/rebuttal_data/rlwrld-rebuttal/<ds>` (rl_common.py:16, drivers) and `/home/nvidia/jonghoon/rebuttal/migrator_data/episodes` (01_calibration, 03a, 04_masks/ego) |19| `/lp-dev/jonghoon/iclr_pose6daug/calib_ws/` — `calibration_solved.json` (hand-eye, DANIILIDIS, `T_ee_egocam`, `T_base_board`, ...), `calib_new_exo.json` + `T_base_exocam_new.npy` (CURRENT exo extrinsic, `variant: flipR`), `time_offset_fit.json`, `ego_exo_transform_frame0.json`, `detections/`, `exo_board/`, `episodes/`, `archive/` (superseded solves, incl. the July `calibration_solved__july-handeye-232ms` and the rlwrld CAD-silhouette fit, and one marked WEAK-KNOWN-BAD) | the live ICLR calibration | every reference to `.../endpoints_ws/calibration_solved.json` (= `/lp-dev/jonghoon/rebuttal_endpoints/calibration_solved.json`; 18 files, e.g. `04_masks/rlwrld_ws/rl_common.py:20`, `04_masks/mask_automation/ego/common.py`, `robot_cad_arm/build_arm_mask_inputs.py`, `03_pose_estimation/lerobot_80eps/{prep_lerobot_batch,postprocess_anchor_eef,propagate_contact_yaw}.py`, `02_gripper_endpoint/*`), to `.../endpoints_ws/calib_20260727_154807/calibration_solved_pair.json` (`compute_ego_poses_lerobot.py`, `render_mesh_overlay_ego_lerobot.py`, `apply_gravity_drop.py`, `overlay_lerobot.py`, `prep_lerobot_batch.py`), and to `{WS}/calib/calibration_rlwrld.json` (`RL_CAL`, rl_common.py:43, drivers) / `0000/T_base_exocam_fitted.json` (`s05_fit_extrinsic.py`). NOTE the schema differs: the rebuttal `calibration_solved_pair.json` bundles `T_base_exocam`+intrinsics+`T_ee_egocam_corrected`; ICLR splits the exo extrinsic into `calib_new_exo.json`/`T_base_exocam_new.npy`. A wrapper must assemble whatever schema each script expects; the rebuttal solves are kept in `01_calibration/solved/` for the schema reference |20| `/lp-dev/jonghoon/iclr_pose6daug/_run/sched.sh` | ICLR async GPU scheduler (job file, one command per line) | the drivers' own fan-out (`render_all_swaps.sh` 18/GPU x 8 GPUs, `pose_rollout.sh`, `run_masks_batch.sh`, `void_ego_gpu47_once.sh` GPUs 4-7) — re-express as job files |21| a NEW per-object workspace, e.g. `/lp-dev/jonghoon/iclr_pose6daug/ws/<object>/` (does not exist yet; pick any writable disk) | frames, depth, events, masks, inpaint, poses, renders | `RL_WS` (39 refs) = `/lp-dev/jonghoon/rebuttal_pose/rlwrld[_<ds>]` (85 files reference `rebuttal_pose` / `pose_ws` / `endpoints_ws`), `pose_ws/lerobot/{g:03d}` (03b, 07, 08), `/lp-dev/jonghoon/rebuttal_pose/{swap_only,swap_all_videos*,swap_all_logs,rollout_logs,locks_global}`, `/lp-dev/jonghoon/rebuttal_vace/{gpu47_jsonl,mask_jobs.txt,ft_jobs.txt,*logs}` |22 23The rlwrld-era scripts already read `RL_*` env overrides (RL_WS, RL_EP_ROOT,24RL_CAL, RL_MESH, RL_POSES, RL_FRAMES(_EGO|_DIR), RL_EVENTS_DIR, RL_EGO_ROBOT,25RL_ROBOT_DIR, RL_OVERLAY_OUT, RL_RELIGHT, RL_DEBUG, RL_ANCHOR_MESH, RL_SEED_*,26RL_DEPTH, RL_INPAINT_DILATE, RL_HOLD_FK, RL_BOX_PROMPTS, RL_HARD_ROBOT_DIR,27RL_NO_FFMPEG — 28 distinct names); the sanctioned re-point is a wrapper `.sh`28that exports them. The lerobot_80eps / endpoint / calibration scripts and the29`migrator_0004_0007` chain have NO env hooks — their paths are literal.30 31## 1. Project root `R` and the regrouped layout32 33`R=/home/nvidia/jonghoon/rebuttal` (symlink to `/lp-dev/jonghoon/rebuttal`) is34baked into every driver and ~150 script lines. The drivers call scripts as35`$R/pose_estimation/*.py`, `$R/overlay/*.py`, `$R/rlwrld/*.py`,36`$R/mask_automation/**`, `$R/rlwrld_rebuttal/scripts/*.py`,37`$R/rlwrld_rebuttal/bash/*.sh` — that per-dir layout is gone from `$R` (the38originals live in `/home/nvidia/jonghoon/rebuttal_storage/`). Mapping to this tree:39 40| original `$R/...` | here |41|---|---|42| `pose_estimation/{build_arm_mask_inputs,dump_arm_masks}.py` | `04_masks/mask_automation/robot_cad_arm/` |43| `pose_estimation/{run_sam3d_*,scale_*,align_red_to_white,render_mesh_views,utils3d_pt_shim}.py` | `06_mesh/` |44| `pose_estimation/*` (rest) | `03_pose_estimation/migrator_0004_0007/` |45| `rlwrld_rebuttal/scripts/*.py` | `03_pose_estimation/lerobot_80eps/`, `07_render_composite/{harmonize_void,render_mesh_overlay,render_mesh_overlay_ego_lerobot,apply_gravity_drop}.py`, `08_swap_video/encode_swap_only.py` |46| `rlwrld_rebuttal/bash/*.sh` | `drivers/` |47| `rlwrld/*` | `04_masks/rlwrld_ws/`, `05_inpainting/void_ego_gpu47_once.sh`, `drivers/{run_episode,run_pair,run_masks_only,run_masks_batch,ft_mask_batch,make_deliverables}.sh` |48| `mask_automation/**` | `04_masks/mask_automation/**`, `05_inpainting/{build_inpaint_masks*,enforce_robot_pixels*}.py` |49| `overlay/*` | `07_render_composite/` |50| `mesh/<obj>/` | `06_mesh/assets/<obj>/` (kanu, green_pear, white_cleanser only) |51| `endpoints_ws/` (= `/lp-dev/jonghoon/rebuttal_endpoints`) | `01_calibration/solved/`, `02_gripper_endpoint/annotations.json` (reference); ICLR live: `../calib_ws/` |52| `.debug/...` | anything writable (`RL_DEBUG`) |53 54## 2. Data roots (36 files)55 56| baked | meaning | ICLR |57|---|---|---|58| `/lp-dev/jonghoon/rebuttal_data/rlwrld-rebuttal/<ds>` | raw 25-ep-per-object captures | `../data/<object>/` (`kanu`, `green_pear`, `white_spray`=white_cleanser, `blue_cup`) |59| `/lp-dev/jonghoon/rebuttal_data/lerobot_eef/franka_rebuttal_eef_80eps_clean` (+ `meta/merged_dataset_manifest.json`, `valid80.json`) | the 80-ep LeRobot export the 03b/07/08 chain indexes by global ep `g` | no ICLR equivalent yet — either export ICLR episodes to the same LeRobot schema or bypass the `g -> (ds,sep)` indirection |60| `/home/nvidia/jonghoon/rebuttal/migrator_data/{episodes,calibration,calib_inputs_*}` | migrator-era ChArUco/h5/svo2 episodes used by 01_calibration and 03a | `../data/calibration/<ep>/` for board captures |61 62## 3. Mesh assets (20 refs)63 64`/home/nvidia/jonghoon/rebuttal/mesh/<obj>/...` and65`/lp-dev/jonghoon/rebuttal_pose/rlwrld/mesh/{white_cleanser_d4.glb,red_cleanser_vc.obj,white_cleanser_d3.glb}`66-> `06_mesh/assets/<obj>/` (only `white_cleanser_d4.glb` of the three67rlwrld/mesh files is here; `red_cleanser_vc.obj` and `white_cleanser_d3.glb`68were not copied — red_cleanser is not an ICLR object). Swap targets used by69`drivers/render_lerobot_swap.sh` (`mango/mesh_metric_w65.glb`,70`abc_chocolate/mesh_metric.glb`, `green_mogi/mesh_metric.glb`) are NOT in this71tree (still at `/lp-dev/jonghoon/rebuttal/video_augmentator/06_mesh/assets/`).72ICLR needs a `blue_cup` mesh (generate with `06_mesh/run_sam3d_textured.py`).73 74## 4. Interpreters (18 files) and external code75 76| baked | env |77|---|---|78| `/home/nvidia/miniconda3/envs/robocasa/bin/python` (10 files) | glue |79| `/data/nvidia/gripper_augmentator/conda-envs/{sam2 (7), any6d (4), vace-wan21 (4), sam3d (1), sam3d_fpose (1)}/bin/python` | SAM2 / Any6D-FoundationPose / VOID / SAM3D |80| `/lp-dev/jonghoon/mv-mesh/envs/reconviagen/bin/python` (6) | nvdiffrast render |81| `/home/nvidia/jonghoon/gripper_augmentator/{vendor/any6d (7), vendor/sam3d (5), gripper_point_tracks (5), inpainting (3)}` | external code; `gripper_point_tracks` + `inpainting` copies are in `vendor/gripper_augmentator/` (same relative layout — a symlink `gripper_augmentator -> scripts/vendor/gripper_augmentator` would satisfy the `sys.path.insert` lines for those two; `vendor/any6d`, `vendor/sam3d` are not vendored) |82| `/lp-dev/jonghoon/mujoco_menagerie/{franka_emika_panda/panda_nohand.xml, robotiq_2f85/2f85.xml, robotiq_2f85_v4/2f85.xml}` | MJCF for the CAD/FK robot masks (`vendor/.../gripper_point_tracks/{render_droid_arm_v1,robotiq_fk}.py`). The tactile-attached ICLR gripper needs its own MJCF/CAD — none is in this tree |83| `/lp-dev/jonghoon/rebuttal_pose/utils3d_pin` (`PYTHONPATH` for `06_mesh/run_sam3d_textured.py`), `/lp-dev/jonghoon/rebuttal_pose/{trt8,cudnn8_links}` (ZED NEURAL depth shim, `s02_extract_depth.sh`) | pinned deps |84| `/home/nvidia/minha/code/sam3d-for-6d` (fallback in `vendor/.../sam3d_mesh.py`), `/home/nvidia/jonghoon/isaac-gr00t` (comment in `gripper_point_tracks`) | neurips-era fallbacks |85 86## 5. Checkpoints / caches (excluded from the copy)87 88| baked | what |89|---|---|90| `/data/nvidia/gripper_augmentator/checkpoints/sam2/sam2.1_hiera_large.pt` (16 refs) | SAM2.1-L |91| `/data/nvidia/gripper_augmentator/checkpoints/sam3d/hf/pipeline.yaml` | SAM3D weights |92| `/data/nvidia/gripper_augmentator/checkpoints/any6d/foundationpose/` | FoundationPose (inside the any6d vendor) |93| `WORKSPACE_CACHE_ROOT`/`VOID_ROOT`/`VOID_CHECKPOINTS` -> `/lp-dev/jonghoon/.cache/gripper_augmentator/{vendor/void,checkpoints/void}` | VOID code (copy in `vendor/gripper_augmentator/vendor/void/`) + CogVideoX-Fun-V1.5-5b-InP + `void_pass1.safetensors` (NOT copied) |94 95## 6. actaug (lineage B) — ICLR-relevant subset96 97`actaug/docs/PORTABILITY.md` is complete. The items that intersect the ICLR98workspace: `/lp-dev/jonghoon/rebuttal_endpoints/calibration_solved.json` (9)99and `annotations.json` (4), `.../calib_20260727_154807/calibration_solved_pair.json`100-> `../calib_ws/`; `/lp-dev/jonghoon/rebuttal_data/lerobot_eef/...` (7) -> ICLR101export; `/lp-dev/jonghoon/sim_action_aug/{code,ws,work}` (~350) -> the actaug102root (a symlink `/lp-dev/jonghoon/sim_action_aug -> .../scripts/actaug` covers the103`code` half only; `ws/`, `work/` are workspaces). The EGL<->CUDA device table baked104into every actaug driver is specific to the rebuttal 8xA100 box.105 106## 7. neurips_robocasa (lineage C)107 108`rendering/render_geometric_augmentation.py` hard-codes `ROOT=/home/nvidia/jonghoon`109and `sys.path` inserts for `gripper_augmentator/{rendering,pose_estimation}` and110imports `render_ep59_teapot_visibility_constrained` from `ROOT`;111`02_inpaint/inpaint_robocasa.sh` defaults `PY=/data/nvidia/gripper_augmentator/conda-envs/vace-wan21/bin/python`112and `VOID_SCRIPT=/home/nvidia/jonghoon/gripper_augmentator/inpainting/run_void_multi.py`113(both overridable via `--py` / `--void_script`). Other baked roots114(`/home/nvidia/jonghoon/robocasa_full`, `/lp-dev/jonghoon/robocasa_full/...`,115`/lp-dev/jonghoon/robocasa_calib/repos`) are robocasa data/repos irrelevant to ICLR.116