MERLIN — MapAnything student
A 230M distilled student of MapAnything that turns a single monocular RGB frame into metric depth + pose, running live at 16-17 FPS (TensorRT INT8) on a $249 Jetson Orin Nano 8GB. Code: github.com/ShayManor/Merlin.
| File | What it is |
|---|---|
student_nf.pt |
Current checkpoint for closed-loop navigation. Near-field fine-tune of student_hab_nav.pt |
student_v3.pt |
Best fidelity checkpoint (0.178 abs_rel vs teacher) |
student_baseline.pt, student_v2.pt |
Uniform-distillation baselines |
student_v2nav1b.pt |
Nav-weighted (M1) variant |
student_earlyexit.pt |
Deep-supervision multi-exit (M2 anytime) |
student_hab.pt, student_hab_nav.pt |
Habitat-finetuned for closed-loop sim |
merlin_student_core.onnx |
ONNX export of the compute core |
merlin_student_core_trt.onnx, merlin_v2_core_trt.onnx |
TRT-ready ONNX (build INT8 engine on the Nano) |
Which checkpoint to use
student_nf.pt for navigation, student_v3.pt for depth fidelity, and note that the two are
not interchangeable.
The navigation failure this model addresses is a data gap, not a modelling one. Training poses were sampled as navigable, so they stand clear of every surface and almost no training pixel sat inside a metre. Fine-tuning on deliberately rendered near-field frames cuts the 0.2-0.5 m signed depth bias from +0.193 m to +0.035 m, and paired closed-loop success on the open-space obstacle benchmark rises from 0.469 to 0.703 (+0.234, 95% CI [+0.047, +0.422], n=32 layouts). Evaluation scenes were held out of fine-tuning.
Two caveats. The 16-17 FPS, INT8 and fidelity figures above were all measured on
student_hab_nav.pt; re-quantize and re-profile before any deployment claim leans on
student_nf.pt. And the closed-loop result is same-simulator, trained and evaluated in
Habitat on ReplicaCAD, so it is not yet evidence the gain survives a real camera.
Longer fine-tunes on the same data were tried and are worse, which is why they are not published here: past ~3000 steps whole-frame accuracy keeps improving while the voxel map degrades, and a 8000-step run blocked 59% of a collision-free route against ground truth's 10%. Select this checkpoint class by the map gate, not by the loss.