WitneyWW/javisdit-edit-both-base-0728-step139000-test10
0
JavisDiT audio-visual object removal - text_both_base_0728 @ step 139000
Ten held-out clips from test.jsonl, each shown four ways: the input, the model's generation, the target the loss was computed against, and the pseudo ground truth the target was built from.
Moved toward the target on 8/10 clips (mean gap L1->target - L1->input = -13.1); 0 silent clips.
What the four columns are
- input - VAE decode of the stored source latent: the clip with the object present.
- generated - the model's edit, prompt
a video without {object}, cfg 1.0, 50 steps, no prepended reference frame (matching training, whose 5000-step anchor warmup ended 134k steps before this checkpoint). - target - VAE decode of the stored target latent, i.e. the exact tensor the training loss used.
- pseudo GT - the raw artifacts that target latent was encoded from: the inpainted video muxed with the SAM-Audio residual track. It is pseudo ground truth because no real "object removed" recording exists - the pair is synthesised by an inpainting model and a source-separation model.
target and pseudo GT are the same content either side of the VAE round trip, so the L1 between them is a floor no generation can beat.
All clips are 256x256, 81 frames @ 16 fps (5.06 s), mono 16 kHz audio.
