blanchon/marigold-v2-ttt-depth-completion

Implements the test-time QLoRA depth completion of Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation (SIGGRAPH Asia 2026), Section 5.1 — a method absent from the official codebase.

Upload an RGB image and a sparse metric depth map (.npy, meters, 0 = no measurement): the Space freezes everything (4-bit NF4 Qwen-Image-Edit-2509 DiT + the frozen rank-128 depth/Log-stage2 depth-prior LoRA), attaches a zero-initialized rank-16 LoRA to the last 12 of 60 transformer blocks (21,233,664 trainable parameters), maps the predicted affine-invariant depth to metric depth with two least-squares-initialized scalars (a, b), and fine-tunes at test time with an L1 + L2 loss at the sparse measurements (Adam, 100 iterations, lr 1e-3 / 3e-2). A single-step rectified-flow pass at t = 499/1000 decodes the depth; the sparse-guidance protocol follows Marigold-DC (Viola et al., ICCV 2025).

Built by reproducing the paper's recipe; validated against the Marigold-DC example data (see results).

Links: paper PDF · model · official code · authors' demo