Flor commited on
Commit
bcfade2
·
verified ·
1 Parent(s): 0af8d36

card: update from fastforks 1cbf1671

Browse files
Files changed (1) hide show
  1. README.md +107 -0
README.md ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: "fastforks"
3
+ pipeline_tag: "image-feature-extraction"
4
+ license: "mit"
5
+ tags:
6
+ - "retrieval"
7
+ - "image-retrieval"
8
+ - "visual-place-recognition"
9
+ - "global-descriptor"
10
+ - "cuda"
11
+ - "pytorch"
12
+ - "fastforks"
13
+ - "fastforks-card"
14
+ ---
15
+ # MegaLoc (fastforks)
16
+
17
+ MegaLoc (Berton and Masone, CVPR 2025 workshops): one image-retrieval model for visual place recognition, landmark retrieval and structure-from-motion pair selection. A DINOv2 ViT-B/14 fine-tuned on a mix of place-recognition, landmark and SfM datasets; its patch features are assigned to 64 clusters by optimal transport (SALAD-style aggregation) and projected to an 8448-d unit descriptor.
18
+
19
+ This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: <https://github.com/kleinicke/fastforks>. Kernels for the `kernels` backend: <https://huggingface.co/fastforks/kernels>. Original code: <https://github.com/gmberton/MegaLoc> at `5fe0dd697c4a`.
20
+
21
+ Author weight repos: <https://huggingface.co/gberton/MegaLoc>.
22
+
23
+ Task: retrieval. Tier: experimental. Outputs: `descriptor`. Backends: `portable`, `kernels`.
24
+
25
+ Input: one RGB image. Output: a global descriptor `[D]` (unit L2 norm) for image retrieval / visual place recognition; `fastforks.retrieval.top_k` and `pairs` find the nearest images (cosine similarity) on the GPU.
26
+
27
+ ## Variants
28
+
29
+ Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in [`card.toml`](https://github.com/kleinicke/fastforks/blob/main/src/fastforks/retrieval/models/megaloc/card.toml).
30
+
31
+ | variant | weights (author link) | SHA-256 | weights licence | commercial use |
32
+ |---|---|---|---|---|
33
+ | `megaloc@1` | [megaloc.safetensors](https://huggingface.co/gberton/MegaLoc/resolve/a0f34722c4297ff787e022433799250180860af7/model.safetensors) | `d4f9f2bcb600` | MIT | yes |
34
+
35
+ ## Licence
36
+
37
+ Original-code strings, recorded as they are (research-use package; check them before any other use). Code: MIT. Licence files: `LICENSE`.
38
+
39
+ Training data: LVD-142M (DINOv2's curated web-image set, not released), GSV-Cities (Google Street View images of 62k places in 40 cities), Mapillary Street-Level Sequences, MegaScenes (Internet photo collections of scenes from Wikimedia Commons), ScanNet (RGB-D scans of indoor scenes), San Francisco eXtra Large (street-view images of San Francisco) (Fine-tuned from DINOv2 ViT-B/14 (LVD-142M) on GSV-Cities, MSLS, MegaScenes, ScanNet and SF-XL (paper, arXiv 2502.17237, Sec. 2); MSLS and ScanNet carry non-commercial terms. Only the weights licence (MIT) counts for the commercial flag.).
40
+
41
+ Licence evidence: https://github.com/gmberton/MegaLoc/blob/5fe0dd697c4a70ba3e23607f6716ab3c606b16db/LICENSE ; https://huggingface.co/gberton/MegaLoc/blob/a0f34722c4297ff787e022433799250180860af7/README.md
42
+
43
+ ## Input size
44
+
45
+ any H x W >= 32, padded to a multiple of 32
46
+ Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `megaloc` 3000x4000 (0.9 GiB peak allocated, 0.9 reserved)
47
+
48
+ ## Speed and memory
49
+
50
+ ### Retrieval, synthetic, 480x640
51
+
52
+ | model | time (ms): original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
53
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- |
54
+ | `megaloc` | 9.68 | 5.29 | 4.26 | 2.3x | 1.2x | 1347 / 1368 | 483 / 508 | 485 / 508 |
55
+
56
+ ### Startup and first call
57
+
58
+ | model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
59
+ | --- | --- | --- | --- | --- | --- | --- |
60
+ | `megaloc` | 2.49 | 0.50 | 1.36 | 0.56 | 0.37 | 0.22 |
61
+
62
+ ### torch.compile and TensorRT
63
+
64
+ | model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
65
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
66
+ | `megaloc` | 480x640 | 5.29 ms | 4.27 ms | 3.21 ms | 4.26 ms | 13.2 s | 51.1 s | 483 / 508 | 518 / 530 | 485 / 508 | 937 / 964 |
67
+
68
+ Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
69
+
70
+ Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastforks/blob/main/docs/benchmarks.md) (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or `(fp32)` where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
71
+
72
+ ## Accuracy
73
+
74
+ ### On public datasets
75
+
76
+ Every query's descriptor against every database descriptor (cosine similarity, exact top-k on the GPU); a query counts as found at k when one of its k nearest database images is within 25 m (UTM positions of the file names), the standard visual place recognition protocol. Original = the original code named under Details, same checkpoint and protocol.
77
+
78
+ | variant | dataset | original fp32 | original fp16 | portable | kernels | note |
79
+ |---|---|---|---|---|---|---|
80
+ | `megaloc` | St Lucia, all 1464 queries against the 1549 database images, recall@1 / @5 % (25 m) | 100.00 / 100.00 | | 100.00 / 100.00 | 100.00 / 100.00 | |
81
+ | `megaloc` | St Lucia, every 5th query (293) against the 1549 database images, recall@1 / @5 % (25 m) | 100.00 / 100.00 | 100.00 / 100.00 | 100.00 / 100.00 | 100.00 / 100.00 | |
82
+
83
+ Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 84333525, 2026-11-13.
84
+
85
+ ### Parity with the original code
86
+
87
+ Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 8.8e-08 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 2.1 % and p99 <= 6.6 % of the reference's median |value| (absolute error of the output `descriptor`).
88
+
89
+ ## Usage
90
+
91
+ ```python
92
+ from fastforks import retrieval
93
+
94
+ model = retrieval.load("megaloc") # mode="auto": the preferred backend from recorded measurements
95
+ result = model("photo.jpg")
96
+ result.descriptor # [D] float32, unit norm
97
+ # fastforks.retrieval.top_k(queries, database, k=5): the nearest database images
98
+ result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
99
+ ```
100
+
101
+ Install with `pip install fastforks` (see <https://github.com/kleinicke/fastforks>). Other variants: `megaloc`.
102
+
103
+ ## Kernels backend
104
+
105
+ `retrieval.load("megaloc", mode="kernels")` runs our CUDA kernels (`fastbb`, `dpt_fused`). It needs a prebuilt kernel for your GPU architecture (downloaded from <https://huggingface.co/fastforks/kernels> on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice.
106
+
107
+ <!-- staged from fastforks 1cbf1671 (2026-11-15) -->