hugging-apps/resplat-recurrent-gs

ReSplat: Learning Recurrent Gaussian Splatting

Interactive demo of **ReSplat** (Haofei Xu, Daniel Barath, Andreas Geiger, Marc Pollefeys) — a feed-forward 3D Gaussian Splatting model that starts from a compact set of Gaussians and then recurrently refines them using its own rendering error, with no per-scene optimisation and no gradient supervision on the update steps.

Upload a set of multi-view images of a room/scene — no poses needed — and the app will

1. 1.run COLMAP structure-from-motion in-Space (pycolmap: SIFT extraction, matching, incremental mapping, undistortion) to estimate intrinsics, camera poses and a sparse point cloud,
2. 2.pick 8 input views by farthest-point sampling over the estimated camera positions,
3. 3.predict a 3D Gaussian scene in a single forward pass,
4. 4.run N recurrent refinement iterations (paper default: 4),
5. 5.show the Gaussians in an interactive 3D viewer you can fly through (orbit / pan / zoom, plus WASD) and export them as .ply,
6. 6.report PSNR/SSIM on the held-out views.

• —Project page: https://haofeixu.github.io/resplat/
• —Code: https://github.com/cvg/resplat (MIT)
• —Weights: https://huggingface.co/haofeixu/resplat (MIT) — this Space runs resplat-base-dl3dv-512x960-view8, the authors' recommended preset.

Implementation notes

• —The inference path is a direct port of the authors' reference script scripts/infer_colmap.py, reusing its COLMAP reader, view selection, pose normalisation and rendering code unchanged.
• —colmap_sfm.py runs the COLMAP pipeline on CPU through pycolmap (feature extraction -> exhaustive/sequential matching -> incremental mapping -> Reconstruction.normalize() -> undistort_images), which writes a standard images/ + sparse/ scene that the reference loader reads as-is. Normalising the reconstruction keeps the world scale consistent with the COLMAP scenes the checkpoint was trained on, so the fixed near/far planes stay meaningful.
• —The interactive viewer is gr.Model3D, which renders Gaussian-splat .ply files with gsplat.js. It is fed a compact copy of the Gaussians (DC colour only, near-transparent splats pruned, re-centred/re-scaled so the default orbit camera frames the scene); the download button serves the full-SH .ply.
• —gsplat's CUDA rasteriser is JIT-compiled at startup against the CUDA toolkit shipped in the ZeroGPU image.
• —The reference implementation also needs pointops, a second hand-written CUDA extension. This Space replaces it with pointops.py, an exact pure-PyTorch reimplementation of the only two ops ReSplat uses (brute-force KNN + neighbour gather), so the numerics are unchanged.

Example scene

The bundled example is a set of 20 raw frames of the authors' official DL3DV demo scene (their COLMAP poses are ignored — the Space re-estimates them). It is downloaded at runtime from the authors' own dataset repo (`haofeixu/depthsplat`) and is not redistributed by this Space. The underlying imagery comes from DL3DV-10K.