disentangled-vla/libero-island-ablation-v1-basevla-bc

V1 BaseVLA (plain BC) — LIBERO Island Eval

Ablation V1 of the Stage3 Qwen2.5-VL framework: BaseVLA-style behavior cloning with no lookahead and no domain disentanglement (single Head A only).

Evaluated on the LIBERO-Spatial island viewpoint protocol from Xing et al. 2025 (CoRL) "Shortcut Learning in Generalist Robot Policies".

Architecture

• —Backbone: Qwen/Qwen2.5-VL-3B-Instruct + LoRA (r=32, q/v_proj)
• —Single AttentiveLatentHead A (8 queries, depth 2, 16 heads, proj_dim=4096)
• —ResNetActionHead (hidden 2048, 2 blocks, chunk=16, action_dim=32 padded to 7-D)
• —Loss: L1(action_pred[..., :7], gt_action[..., :7]) only
• —No InfoNCE / no SIGReg / no V-JEPA target / no augmentation

Training

• —Data: LIBERO-Spatial island A+B paper-200 protocol (~211K frames, 1,724 eps)
• —bs=64 single GPU, lr=1e-4 cosine, 100K steps, 10 epochs
• —Final loss: ≈0.06

Evaluation protocol

6 settings × 5 tasks × 5 trials = 25 episodes / setting (paper "paper-200" sub-sample). Per-episode random viewpoint angle sampled from the setting's range. Max 220 steps / episode; closed-loop with 8 open-loop steps before re-query.

Results — epoch_10

Plain BC overfits the two training islands and fails completely on cross-domain viewpoints — the canonical shortcut-learning failure mode.

Files

eval_ablation_v1_first_videos/
├── {setting}_results.jsonl     # per-episode results (success, ep length, etc.)
├── {setting}_summary.json      # overall SR + per-task breakdown
└── videos/{setting}/task{N}_trial{T}_a{angle}_{OK,FAIL}.mp4

Videos: 25 ep × 6 settings = 150 mp4 total.