CoolFace
Apppublic

WitneyWW/javisdit-edit-both-base-0728-step139000-test10

sourceHugging Faceupdated 2mo agoView on Hugging Face
0likes
App README

JavisDiT audio-visual object removal - text_both_base_0728 @ step 139000

Ten held-out clips from test.jsonl, each shown four ways: the input, the model's generation, the target the loss was computed against, and the pseudo ground truth the target was built from.

mean L1 (0-255)
generated -> target19.2
generated -> input32.3
generated -> pseudo GT19.5
target -> pseudo GT (VAE floor)4.0

Moved toward the target on 8/10 clips (mean gap L1->target - L1->input = -13.1); 0 silent clips.

What the four columns are

  • —input - VAE decode of the stored source latent: the clip with the object present.
  • —generated - the model's edit, prompt a video without {object}, cfg 1.0, 50 steps, no prepended reference frame (matching training, whose 5000-step anchor warmup ended 134k steps before this checkpoint).
  • —target - VAE decode of the stored target latent, i.e. the exact tensor the training loss used.
  • —pseudo GT - the raw artifacts that target latent was encoded from: the inpainted video muxed with the SAM-Audio residual track. It is pseudo ground truth because no real "object removed" recording exists - the pair is synthesised by an inpainting model and a source-separation model.

target and pseudo GT are the same content either side of the VAE round trip, so the L1 between them is a floor no generation can beat.

All clips are 256x256, 81 frames @ 16 fps (5.06 s), mono 16 kHz audio.