WitneyWW/history-mot-nodf-rollouts

History-conditioned MoT (no-DF ablation): inference rollouts, continued training

Ground-truth vs. rollout videos, GT-vs-prediction frame strips, and metrics for mot_hist_vt_actionimage/eval_e129 — the no-diffusion-forcing ablation of the DF action-image world model, evaluated at its best continued-training checkpoint (epoch 129).

The model

Identical to mot_df_vt_actionimage (two-expert MoT, action-image + cross-attention conditioning, actionflow prior, data, seed, optimizer) except the noising scheme:

• —Training: the 2 history latent frames stay exactly clean (level 0); all future frames share ONE sampled noise level per modality; loss on future frames only.
• —Sampling: full_sequence — the whole future denoised jointly, no per-frame noise-level schedule.
• —Continued training: resumed from epoch 100 with max_epochs 1000 and early stopping (patience 15 on val_loss); stopped at epoch 144, best at epoch 129 (val 0.2249 vs 0.2271 at epoch 76).

Result

Diffusion forcing beats this model in every cell — including short-horizon prediction, the exact task this model trains on (short test tactile: 36.35 vs 44.62 dB). The continuation recovered ~+0.3 dB short / ~+1.0 dB long tactile over its own epoch-100 checkpoint but did not close the gap. DF's mixed-noise curriculum acts as strong augmentation/regularization, not just long-rollout robustness training.

Conventions

Same 12 (episode, fps6-start) rows as the companion Spaces (actionimage-mot-world-model-rollouts, visuotactile-mot-world-model-rollouts). Context = fps6 frames [start, start+8); prediction starts at fps6 frame start+8 (decoded frame 5 of the 13-frame clips — the red line in the strips).