Apppublic

hugging-apps/aq3d-instance-segmentation

sourceHugging Faceupdated 20d agoView on Hugging Face

0likes

App README

AQ3D — Adaptive Query Transformer for 3D Instance Segmentation

Upload an indoor surface mesh (.ply / .obj / .glb) and AQ3D labels every object in it with one of the 198 ScanNet200 instance classes.

—Paper: AQ3D: Adaptive Query Transformer for 3D Instance Segmentation
—Code: kenomo/aq3d · Weights: kenomo/aq3d

This Space runs the `aq3d_scannet200_volt` checkpoint (Volt-B point transformer backbone + AQTD adaptive-query decoder, 44.1 mAP on ScanNet200 val).

How the pipeline is reproduced

The upstream inference stack depends on several compiled extensions that have no wheels for the Space's Blackwell GPU (segmentator, torch_scatter, torch_geometric, flash-attn). They are replaced here by exact, dependency-free equivalents:

upstream	here
`segmentator.segment_mesh` (C++/libtorch Felzenszwalb–Huttenlocher)	Numba port in `superpoints.py` (same edge order, weights, two-pass union–find)
`torch_scatter` / `torch_geometric.nn.pool.fps`	`nnutils.py` (`scatter_*` via `scatter_add_`/`scatter_reduce_`, greedy FPS)
`flash_attn_varlen_qkvpacked_func`	`F.scaled_dot_product_attention` per sequence (identical for batch size 1)
`timm` blocks	inlined (`Mlp`, identity LayerScale/DropPath at eval)

Everything else — MeanCoord → NormalizeColor(-1, 1) → GridSample(0.02) with FNV hashing, superpoint attention pooling, superpoint NMS (0.8), adaptive top-k, mask scoring, and the point-count threshold — follows the official ScanNet200 validation configuration verbatim.

Example scans

The bundled examples are CC BY 4.0 photogrammetry / LiDAR room scans from Objaverse, redistributed via Zenodo:

—Attic — "my room and the mess therein", Zenodo record 10380976 — CC BY 4.0
—Historic building interior — iPhone LiDAR capture, Zenodo record 10325220 — CC BY 4.0

Caveats

AQ3D is trained on ScanNet200, i.e. RGB-D reconstructions of real rooms in metres with Z up. Meshes far outside that domain (synthetic scenes, single objects, non-metric units) will still run but the labels will be unreliable. Use the Scene orientation & scale accordion if the automatic guess is wrong.