Update README.md
Browse files
README.md
CHANGED
|
@@ -41,22 +41,8 @@ Global
|
|
| 41 |
|
| 42 |
### Release Date:
|
| 43 |
|
| 44 |
-
Hugging Face
|
| 45 |
-
|
| 46 |
-
## References(s):
|
| 47 |
-
|
| 48 |
-
- *MetricTrail: Revisiting the Power of Metric Depth and Local Optimization for
|
| 49 |
-
Long-Sequence Streaming 3D Reconstruction*.
|
| 50 |
-
- [Repository overview and running instructions](../README.md).
|
| 51 |
-
- [DPVO](https://github.com/princeton-vl/DPVO), the sparse local odometry backbone.
|
| 52 |
-
- [DINOv3](https://github.com/facebookresearch/dinov3) and the
|
| 53 |
-
[ViT-S/16 LVD-1689M pretrained model](https://huggingface.co/facebook/dinov3-vits16-pretrain-lvd1689m).
|
| 54 |
-
- [MoGe](https://github.com/microsoft/MoGe) and
|
| 55 |
-
[MoGe-2](https://arxiv.org/abs/2507.02546), the default metric-depth provider.
|
| 56 |
-
- [SALAD](https://github.com/serizba/salad), an optional external place-retrieval
|
| 57 |
-
tool obtained and run independently of MetricTrail.
|
| 58 |
-
- [ViPE](https://github.com/nv-tlabs/vipe), used for RGB-only shared-focal calibration.
|
| 59 |
-
- [RoMa](https://github.com/Parskatt/RoMa), used for optional revisit-based focal refinement.
|
| 60 |
|
| 61 |
## Model Architecture:
|
| 62 |
|
|
@@ -101,52 +87,6 @@ uses 80 patches per frame and 18 recurrent refinements. The frozen DINOv3
|
|
| 101 |
forward pass uses BF16; adapters, odometry, differentiable bundle adjustment,
|
| 102 |
and losses use FP32.
|
| 103 |
|
| 104 |
-
<!--
|
| 105 |
-
## Computational Load (internal review only)
|
| 106 |
-
|
| 107 |
-
Remove this entire comment from public source and the published model card.
|
| 108 |
-
|
| 109 |
-
Scope: added compute for the documented MetricTrail adaptation run, including
|
| 110 |
-
fusion warmup and joint fine-tuning. The run uses 16 A100 80GB GPUs for
|
| 111 |
-
approximately 7 hours (112 GPU-hours), processing 240,000 clips of 15 frames at
|
| 112 |
-
480 x 640 with 80 patches per frame and 18 recurrent refinements. Third-party
|
| 113 |
-
DPVO and DINOv3 pretraining, evaluation, and discarded exploratory runs are
|
| 114 |
-
excluded. MoGe-2 is not used during adaptation.
|
| 115 |
-
|
| 116 |
-
**Cumulative Compute (estimated): 1.3 x 10^18 FLOPs.**
|
| 117 |
-
This is an analytical operation estimate, not a hardware-profiler measurement.
|
| 118 |
-
Counting a multiply-add as two FLOPs gives approximately 2.8 x 10^17 FLOPs for
|
| 119 |
-
frozen DINOv3 forward passes, 2.5 x 10^17 for the DPVO convolutional encoders,
|
| 120 |
-
and 6.9 x 10^17 for recurrent-update matrix operations. The encoder/update
|
| 121 |
-
counts approximate forward plus backward as three times forward, with
|
| 122 |
-
adjustments for the frozen warmup. The subtotal is 1.22 x 10^18 FLOPs; the
|
| 123 |
-
reported value includes an approximate allowance for correlation, adapters,
|
| 124 |
-
bundle adjustment, losses, and other operations. Random edge pruning and
|
| 125 |
-
kernel details are not reconstructed. A reasonable sensitivity range is
|
| 126 |
-
0.8 x 10^18 to 2.0 x 10^18 FLOPs, not a validated confidence interval.
|
| 127 |
-
Sources for the operation count: DPVO/dpvo/net.py, DPVO/dpvo/extractor.py,
|
| 128 |
-
DPVO/dpvo/blocks.py, and metrictrail/runtime/dino_fusion.py.
|
| 129 |
-
|
| 130 |
-
**Estimated Energy for Model Training: 50 kWh.**
|
| 131 |
-
An ORD hardware query on 10/20/2026 confirmed NVIDIA A100-SXM4-80GB with
|
| 132 |
-
configured and default power limits of 400 W. Applying this rating to the
|
| 133 |
-
documented training run, the prescribed calculation is
|
| 134 |
-
16 x 0.400 kW x 0.80 x 7 hours x 1.4 PUE = 50.176 kWh.
|
| 135 |
-
The 80% power fraction and PUE of 1.4 are the internal guidance defaults,
|
| 136 |
-
not measured power or a verified ORD facility PUE.
|
| 137 |
-
|
| 138 |
-
**Estimated Emissions for Model Training: 0.021 tCO2e (approximately 21 kgCO2e).**
|
| 139 |
-
50.176 kWh x 410.5 gCO2e/kWh x 0.000001 = 0.020597 tCO2e.
|
| 140 |
-
The emissions factor is the guidance's fallback value; a site-specific
|
| 141 |
-
electricity emissions factor was not available. These are estimates for
|
| 142 |
-
the documented adaptation run, not metered energy or emissions.
|
| 143 |
-
|
| 144 |
-
Methods:
|
| 145 |
-
https://nvidia.sharepoint.com/sites/TrustworthyAI/SitePages/How-to-Assess-Training-Compute.aspx
|
| 146 |
-
https://nvidia.sharepoint.com/:w:/s/TrustworthyAI/Ecb23D6b-kNLv1c6LCdhSiMBjWFz0MKncBaRcryDsfv-vA?e=lIcO9z
|
| 147 |
-
Hardware specifications: https://www.nvidia.com/en-us/data-center/a100/
|
| 148 |
-
-->
|
| 149 |
-
|
| 150 |
## Input:
|
| 151 |
|
| 152 |
**Input Type(s):** A temporally ordered RGB image sequence and camera parameters;
|
|
@@ -156,9 +96,7 @@ configuration selects the data adapter and processing settings.
|
|
| 156 |
**Input Format(s):** Prepared PNG frames or KITTI NPZ files containing `images`
|
| 157 |
as `uint8 [N,3,H,W]`. The repository also supplies Oxford Spires and LaMAria
|
| 158 |
inventory adapters. Loaders produce CHW RGB `float32` images in `[0,1]`.
|
| 159 |
-
Arbitrary video containers are not decoded by this CLI.
|
| 160 |
-
[input layouts](../README.md#layout-of-external-files) and
|
| 161 |
-
[data adapters](../metrictrail/datasets.py).
|
| 162 |
|
| 163 |
Optional loop correction consumes a saved local trajectory NPZ, its successful
|
| 164 |
JSON sidecar, and an externally generated candidate JSON containing frame pairs
|
|
@@ -263,9 +201,7 @@ metadata permits broader versions, but those combinations have not been validate
|
|
| 263 |
No precompiled DPVO CUDA extensions are included. Before tracking, build
|
| 264 |
`cuda_corr`, `cuda_ba`, and `lietorch_backends`
|
| 265 |
from source in the inference environment, using a compatible C++ compiler,
|
| 266 |
-
the CUDA toolkit including `nvcc`, and Eigen 3.4.0 headers.
|
| 267 |
-
[environment instructions](../README.md#environment)
|
| 268 |
-
and [package dependencies](../pyproject.toml).
|
| 269 |
|
| 270 |
**Supported Hardware Microarchitecture Compatibility:**
|
| 271 |
|
|
@@ -283,7 +219,7 @@ Run with `PYTHONNOUSERSITE=1`. The primary command is
|
|
| 283 |
`cuda:0`. `--case`, `--resume`, `--preflight`, and
|
| 284 |
`--stop-after {calibration,local}` control selection and stage execution.
|
| 285 |
Preflight checks external files and relevant runtime imports; it does not
|
| 286 |
-
validate model quality.
|
| 287 |
|
| 288 |
The main pipeline ends at the local trajectory with `enable_lc: false`.
|
| 289 |
Run optional loop correction separately on CPU with
|
|
@@ -328,7 +264,7 @@ obtained separately as `dinov3_vits16_pretrain_lvd1689m-08c60483.pth`.
|
|
| 328 |
Its validated SHA256 is
|
| 329 |
`08c60483bc63c04f533611e34bf70b120eedb7240f469bc16e9e20bf344b941d`.
|
| 330 |
MoGe-2 and optional calibration and RoMa dependencies are also separate
|
| 331 |
-
downloads
|
| 332 |
Optional loop-retrieval tools are obtained and run independently to generate
|
| 333 |
the candidate files consumed by the loop-correction command.
|
| 334 |
|
|
@@ -462,8 +398,7 @@ results use meters; additional indoor/dynamic trajectory results use centimeters
|
|
| 462 |
Dataset averages give sequence-level results equal weight in the reported
|
| 463 |
additional protocols. LC denotes the original SALAD/PyPose loop-closure workflow
|
| 464 |
used in our evaluations. These results do not establish numerical or benchmark
|
| 465 |
-
equivalence for the new external-candidate NumPy/SciPy postprocessor.
|
| 466 |
-
[the loop-correction workflow](../docs/loop-closure.md).
|
| 467 |
|
| 468 |
| Benchmark / configuration | Average ATE |
|
| 469 |
| --- | ---: |
|
|
|
|
| 41 |
|
| 42 |
### Release Date:
|
| 43 |
|
| 44 |
+
Hugging Face 11/05/2026 via [https://huggingface.co/nvidia/metrictrail](https://huggingface.co/nvidia/metrictrail)
|
| 45 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
## Model Architecture:
|
| 48 |
|
|
|
|
| 87 |
forward pass uses BF16; adapters, odometry, differentiable bundle adjustment,
|
| 88 |
and losses use FP32.
|
| 89 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
## Input:
|
| 91 |
|
| 92 |
**Input Type(s):** A temporally ordered RGB image sequence and camera parameters;
|
|
|
|
| 96 |
**Input Format(s):** Prepared PNG frames or KITTI NPZ files containing `images`
|
| 97 |
as `uint8 [N,3,H,W]`. The repository also supplies Oxford Spires and LaMAria
|
| 98 |
inventory adapters. Loaders produce CHW RGB `float32` images in `[0,1]`.
|
| 99 |
+
Arbitrary video containers are not decoded by this CLI.
|
|
|
|
|
|
|
| 100 |
|
| 101 |
Optional loop correction consumes a saved local trajectory NPZ, its successful
|
| 102 |
JSON sidecar, and an externally generated candidate JSON containing frame pairs
|
|
|
|
| 201 |
No precompiled DPVO CUDA extensions are included. Before tracking, build
|
| 202 |
`cuda_corr`, `cuda_ba`, and `lietorch_backends`
|
| 203 |
from source in the inference environment, using a compatible C++ compiler,
|
| 204 |
+
the CUDA toolkit including `nvcc`, and Eigen 3.4.0 headers.
|
|
|
|
|
|
|
| 205 |
|
| 206 |
**Supported Hardware Microarchitecture Compatibility:**
|
| 207 |
|
|
|
|
| 219 |
`cuda:0`. `--case`, `--resume`, `--preflight`, and
|
| 220 |
`--stop-after {calibration,local}` control selection and stage execution.
|
| 221 |
Preflight checks external files and relevant runtime imports; it does not
|
| 222 |
+
validate model quality.
|
| 223 |
|
| 224 |
The main pipeline ends at the local trajectory with `enable_lc: false`.
|
| 225 |
Run optional loop correction separately on CPU with
|
|
|
|
| 264 |
Its validated SHA256 is
|
| 265 |
`08c60483bc63c04f533611e34bf70b120eedb7240f469bc16e9e20bf344b941d`.
|
| 266 |
MoGe-2 and optional calibration and RoMa dependencies are also separate
|
| 267 |
+
downloads.
|
| 268 |
Optional loop-retrieval tools are obtained and run independently to generate
|
| 269 |
the candidate files consumed by the loop-correction command.
|
| 270 |
|
|
|
|
| 398 |
Dataset averages give sequence-level results equal weight in the reported
|
| 399 |
additional protocols. LC denotes the original SALAD/PyPose loop-closure workflow
|
| 400 |
used in our evaluations. These results do not establish numerical or benchmark
|
| 401 |
+
equivalence for the new external-candidate NumPy/SciPy postprocessor.
|
|
|
|
| 402 |
|
| 403 |
| Benchmark / configuration | Average ATE |
|
| 404 |
| --- | ---: |
|