hchen657Naza/robotwin_dream_success_videos

robotwindreamsuccess_videos

DREAMED (world-model-imagined) videos judged SUCCESS BY ROBOMETER (not by the sim), with actions. Layout mirrors robotwinrealsuccessvideos: `<task>/<scene>/set1/episodeN/{dream.mp4, sidebyside.mp4, actions.npz}` (set2, set3, ... later). Episode numbers keep source indices; only robometer keep==1 episodes included. dream.mp4 = the WM-imagined rollout; sidebyside.mp4 = GT | dream | skeleton composite; no standalone real video.

set1 (2026-09-01), Wide_D435 fovy-55:

• —adjustbottle (98 clean + 98 noisy) and liftpot (91 clean + 32 noisy): ACT-policy dreams inside the history/action-skeleton WM (v2a lorastep4000); robometer succfinal with DREAM-TUNED thresholds (adjustbottle 0.083, liftpot 0.215). actions.npz also carries the GT sim_success flag.
• —moveplayingcardaway (82), clickbell (104), placecontainerplate (61), stackblockstwo (10): noisy cap-x dreams (capxdreamdemo); robometer per-task best metric with REAL-VIDEO-calibrated thresholds (2026-08-27 filtering, robometerfilter/capxdreamsfiltered.csv in capxdreamdemo). CAVEAT: robometer aligns only weakly with GT sim success on dreams (AUC 0.54-0.64, "dream optimism") — see capxdreamdemo/robometerseparationcapx3.md and actdreams/READMErobometer_filter.md locally. Total: 576 episodes.

set2+ (2026-09-02), WideD435 fovy-55, cap-x single-turn dreams from the 8-GPU collection run (WM = ckptcapx7v2aaskelhist5lora29/lorastep4000, 5-frame sparse history, target-action skeletons):

• —placecontainerplate clean set2 (47), noisy set2 (19)
• —shake_bottle noisy set2 (49), set3 (50)
• —placeobjectscale noisy set2 (9), set3 (4), set4 (21)
• —placebreadbasket noisy set2 (9), set3 (8) 216 of 480 collected dreams, kept by robometer succfinal with REAL-VIDEO-calibrated per-task thresholds from calibcapx10.csv (40 success + 25 failed per task): placecontainerplate 0.122 (calib AUC 1.00), placeobjectscale 0.547 (0.88), placebreadbasket 0.910 (0.69), shakebottle 0.034 (0.64). One set per source collector process (disjoint scene seeds: set2 = seeds 400001+, set3 = 50001+, set4 = 700001+); episode numbers are the source indices within that process. `metadataset2plus.csv lists task, scene, set, episode, seed, robometer metric/score/threshold, the GT simsuccess` flag and frame count for every episode here. CAVEAT (set2+): only placecontainerplate and placeobjectscale separate meaningfully. shakebottle's threshold (0.034, AUC 0.64) passes 99 of 100 dreams, and placebreadbasket's (0.910, AUC 0.69) sits above its own calibration success mean and rejects most true successes (agrees with GT sim on 5 of 48) — treat those two tasks' "success" labels here as unfiltered.

set2..set6 (2026-09-03), WideD435 fovy-55, stackblockstwo cap-x single-turn dreams from the 7-worker collection on the L40S box (WM = ckptcapx10askelhist5lora29/lorastep2500, 5-frame sparse history, TARGET-ACTION skeletons -- collectcapxdream.py was fixed on 2026-09-02 to render its conditioning skeletons from the commanded 14-D targets instead of the measured link poses):

• —stackblockstwo clean set2 (21), set3 (25); noisy set2 (18), set3 (14), set4 (19), set5 (16), set6 (18) 131 of 184 collected dreams (71%), kept by robometer succfinal >= 0.531 -- the REAL-VIDEO-calibrated stackblockstwo threshold from calibcapx10.csv (40 success + 25 failed, AUC 0.99, balanced acc 0.98), the strongest calibration of the ten tasks after placecontainerplate. Agreement with GT sim success on this batch: 76% overall; of the 131 kept dreams 114 (87%) also succeeded in sim, and the filter caught 81% of the sim-successful episodes. Pass rate is much higher on clean (46/50 = 92%) than noisy (85/134 = 63%). The usual dream-optimism caveat still applies: robometer scores what the WM RENDERED, not what the actions would really achieve. One set per source collector process, disjoint seed bases: clean set2 = 813400+, set3 = 824700+; noisy set2 = 836100+, set3 = 847900+, set4 = 859300+, set5 = 861700+, set6 = 872900+. metadata_stack_blocks_two_capx10.csv lists task, scene, set, episode, seed, robometer metric/score/ threshold, the GT sim_success flag and frame count. Full unfiltered scores for all 184 dreams are in sbt_dreams.csv / sbt_dreams_filtered.csv.