Modelpublic

ShayManor/merlin-mapanything-student

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

1likes

Model Card

MERLIN — MapAnything student

A 230M distilled student of MapAnything that turns a single monocular RGB frame into metric depth + pose, running live at 16-17 FPS (TensorRT INT8) on a $249 Jetson Orin Nano 8GB. Code: github.com/ShayManor/Merlin.

File	What it is
`student_nf.pt`	Current checkpoint for closed-loop navigation. Near-field fine-tune of `student_hab_nav.pt`
`student_v3.pt`	Best fidelity checkpoint (0.178 abs_rel vs teacher)
`student_baseline.pt`, `student_v2.pt`	Uniform-distillation baselines
`student_v2nav1b.pt`	Nav-weighted (M1) variant
`student_earlyexit.pt`	Deep-supervision multi-exit (M2 anytime)
`student_hab.pt`, `student_hab_nav.pt`	Habitat-finetuned for closed-loop sim
`merlin_student_core.onnx`	ONNX export of the compute core
`merlin_student_core_trt.onnx`, `merlin_v2_core_trt.onnx`	TRT-ready ONNX (build INT8 engine on the Nano)

Which checkpoint to use

student_nf.pt for navigation, student_v3.pt for depth fidelity, and note that the two are not interchangeable.

The navigation failure this model addresses is a data gap, not a modelling one. Training poses were sampled as navigable, so they stand clear of every surface and almost no training pixel sat inside a metre. Fine-tuning on deliberately rendered near-field frames cuts the 0.2-0.5 m signed depth bias from +0.193 m to +0.035 m, and paired closed-loop success on the open-space obstacle benchmark rises from 0.469 to 0.703 (+0.234, 95% CI [+0.047, +0.422], n=32 layouts). Evaluation scenes were held out of fine-tuning.

Two caveats. The 16-17 FPS, INT8 and fidelity figures above were all measured on student_hab_nav.pt; re-quantize and re-profile before any deployment claim leans on student_nf.pt. And the closed-loop result is same-simulator, trained and evaluated in Habitat on ReplicaCAD, so it is not yet evidence the gain survives a real camera.

Longer fine-tunes on the same data were tried and are worse, which is why they are not published here: past ~3000 steps whole-frame accuracy keeps improving while the voxel map degrades, and a 8000-step run blocked 59% of a collision-free route against ground truth's 10%. Select this checkpoint class by the map gate, not by the loss.