CoolFace
Datasetpublic

iug8oyo8/IMS2026-HW0-media

IMS2026 HW0 — media Videos, checkpoint and evaluation artifacts for github.com/henry2-9/IMS2026-HW0. They live here rather than in the repository because the assignment asks for datasets, checkpoints and videos to be shared via the Hub or a cloud drive. task1_ticvla_dynanav/ — TIC-VLA on DynaNav 8 episodes in Isaac Sim 5.0.0 covering four scenes x two robot platforms. Each clip is side by side: the robot's RGB camera left, third-person right. H.264, so they play… See the full description on the dataset page: https://huggingface.co/datasets/iug8oyo8/IMS2026-HW0-media.

sourceHugging Facemitupdated 12d agoView on Hugging Face
0likes222downloads
Dataset Card

IMS2026 HW0 — media

Videos, checkpoint and evaluation artifacts for github.com/henry2-9/IMS2026-HW0. They live here rather than in the repository because the assignment asks for datasets, checkpoints and videos to be shared via the Hub or a cloud drive.

task1_ticvla_dynanav/ — TIC-VLA on DynaNav

8 episodes in Isaac Sim 5.0.0 covering four scenes x two robot platforms. Each clip is side by side: the robot's RGB camera left, third-person right. H.264, so they play in any browser.

fileplatformsceneoutcome
episode_1_carter.mp4Nova Carterhospitalsuccess, 18.5 s
episode_2_carter.mp4Nova Carterwarehousetimeout
episode_3_carter.mp4Nova Carterofficesuccess, 21.2 s
episode_4_carter.mp4Nova Carteroutdoor, 200 peoplesuccess, 23.4 s
episode_5_spot.mp4Spothospitalfail, NE 22.95 m
episode_6_spot.mp4Spotwarehousesuccess, 58.6 s
episode_7_spot.mp4Spotofficefail, NE 6.96 m
episode_8_spot.mp4Spotoutdoor, 200 peopletimeout

Aggregate: 50.0 % success, 0.0 % collision, 9.45 m avg navigation error, 0.440 SPL.

task2_smolvla_libero/ — SmolVLA on LIBERO

  • —pretrained_model/ — the submitted checkpoint. SmolVLA 0.450 B, 100k steps, batch 64, scheduler_decay_steps=100000. Load it with lerobot-eval --policy.path=....
  • —live_view_400ep/ — live-view recording of every one of the 400 evaluation episodes. Each frame carries the front and wrist camera panes, the suite and instruction, the episode and step counters, and the running success rate.
  • —eval_info.json — per-suite and per-task results with compute logs.

Three-seed means (1200 episodes), n_action_steps=10, `mujoco<3.4.0`:

SuiteOursPaperwithin ±3 pp
LIBERO-Spatial86.7 %90 %no, by 0.3 pp
LIBERO-Object94.7 %96 %yes
LIBERO-Goal91.3 %92 %yes
LIBERO-Long74.7 %71 %no — overshoots
Average86.8 %87.3 %−0.5

Every suite lands within 3.7 pp of the paper.

The MuJoCo pin matters. MuJoCo 3.4.0 fixed a box-box collision bug that LIBERO's stored initial state for libero_spatial task 5 depended on: the bowl no longer settles onto the ramekin and ends up tilted on its rim. That task drops from 70 % to 33 %, costing ~6 pp on the suite. Nothing in lerobot, robosuite or hf-libero caps the version — see huggingface/lerobot#4390. The recordings here were produced with mujoco 3.3.7.