Realcat commited on
Commit
97a137d
·
verified ·
1 Parent(s): 77e55f4

Delete megaloc/README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. megaloc/README.md +0 -148
megaloc/README.md DELETED
@@ -1,148 +0,0 @@
1
- # MegaLoc Quantized Models
2
-
3
- Quantized ONNX and CoreML models for [MegaLoc](https://arxiv.org/abs/2502.17237) — a state-of-the-art visual place recognition model based on DINOv2 ViT-B/14.
4
-
5
- All benchmarks below were run on a MacBook Pro with Apple Silicon (M-series), 518×518 input, single-threaded CPU for ONNX Runtime, GPU for CoreML.
6
-
7
- ## Available Models
8
-
9
- | File | Format | Size | Ratio | CosSim | Latency | FPS | Notes |
10
- |------|--------|------|-------|--------|---------|-----|-------|
11
- | `megaloc.onnx` | ONNX FP32 | 873 MB | 1.0× | 1.000000 | 387 ms | 2.6 | Baseline, float32 reference |
12
- | `megaloc_fp16.onnx` | ONNX FP16 | 437 MB | 2.0× | 0.999988 | 394 ms | 2.5 | Near-lossless, recommended for ONNX |
13
- | `megaloc_int8.onnx` | ONNX INT8 | 223 MB | 3.9× | 0.996453 | 475 ms | 2.1 | Weight-only quantization, Recall@1 still 100% |
14
- | `megaloc_coreml_fp32.mlpackage` | CoreML FP32 | 873 MB | 1.0× | 1.000000 | 78 ms | 12.8 | CoreML baseline, 5× faster than ONNX |
15
- | `megaloc_coreml_int8.mlpackage` | CoreML INT8 | 219 MB | 4.0× | 0.998315 | 78 ms | 12.9 | **Recommended** — best size/speed/accuracy trade-off |
16
-
17
- ## Benchmark Summary
18
-
19
- ```
20
- Model Size Ratio CosSim Latency FPS
21
- ─────────────────────────────────────────────────────────────────────────
22
- ONNX FP32 872.9 MB 1.0× 1.000000 387 ms 2.6
23
- ONNX FP16 436.8 MB 2.0× 0.999988 394 ms 2.5
24
- ONNX INT8 222.5 MB 3.9× 0.996453 475 ms 2.1
25
- CoreML FP32 872.8 MB 1.0× 1.000000 78 ms 12.8
26
- CoreML INT8 218.8 MB 4.0× 0.998315 78 ms 12.9
27
- ```
28
-
29
- ### Retrieval Accuracy
30
-
31
- | Model | Recall@1 | Recall@3 | Recall@5 | Kendall Tau |
32
- |-------|----------|----------|----------|-------------|
33
- | ONNX FP16 | 100% | 100% | 100% | 0.94 |
34
- | ONNX INT8 | 100% | 100% | 100% | 0.58 |
35
-
36
- Despite a drop in descriptor-level cosine similarity, INT8 models preserve top-K retrieval perfectly. The full ranking order (Kendall Tau) shows some reshuffling, but the most relevant results remain correct.
37
-
38
- ### Why CoreML is faster than ONNX Runtime?
39
-
40
- - ONNX Runtime uses CPU execution by default on macOS, while **CoreML leverages Apple Silicon GPU** via Metal Performance Shaders
41
- - ONNX INT8 is slower (475 ms vs 387 ms) because the weight-only quantization inserts `DequantizeLinear` nodes — each MatMul first converts INT8 back to float32, adding overhead without the speed benefit of integer math
42
- - CoreML INT8 runs at the same speed as FP32 (78 ms) because the GPU's integer compute units are used natively
43
-
44
- ### On sub-8-bit quantization
45
-
46
- We experimented with 4-bit and 1-bit quantization (ONNX INT4, CoreML 4-bit/2-bit K-Means palettization). **All failed** with cosine similarity ≈ 0. The DINOv2 ViT backbone has 12 transformer blocks + aggregator (~200 weight tensors). Quantization error from only 16 levels accumulates catastrophically across 48+ MatMul layers. This is a known limitation — ViT architectures below 8-bit require **quantization-aware training (QAT)** to preserve accuracy.
47
-
48
- ## Usage
49
-
50
- ### ONNX Inference
51
-
52
- ```bash
53
- pip install onnxruntime pillow numpy
54
-
55
- python3 -c "
56
- import onnxruntime as ort, numpy as np
57
- from PIL import Image
58
-
59
- # Choose from the table above
60
- session = ort.InferenceSession('megaloc_fp16.onnx', providers=['CPUExecutionProvider'])
61
-
62
- # Preprocess: resize to 518×518, normalize to [0, 1]
63
- img = Image.open('image.jpg').convert('RGB').resize((518, 518), Image.LANCZOS)
64
- data = np.array(img).astype(np.float32) / 255.0
65
- data = np.expand_dims(data.transpose(2, 0, 1), axis=0) # [1, 3, 518, 518]
66
-
67
- # Inference → L2-normalized descriptor [1, 8448]
68
- descriptor = session.run(None, {'images': data})[0]
69
-
70
- # Compare two images via cosine similarity
71
- cos_sim = np.dot(desc1, desc2.T) # both already L2-normalized
72
- "
73
- ```
74
-
75
- ### CoreML Inference (macOS only)
76
-
77
- ```bash
78
- pip install coremltools pillow numpy
79
-
80
- python3 -c "
81
- import coremltools as ct, numpy as np
82
- from PIL import Image
83
-
84
- model = ct.models.MLModel('megaloc_coreml_int8.mlpackage')
85
-
86
- # Preprocess
87
- img = Image.open('image.jpg').convert('RGB').resize((518, 518), Image.LANCZOS)
88
- data = np.array(img).astype(np.float32) / 255.0
89
- data = np.expand_dims(data.transpose(2, 0, 1), axis=0)
90
-
91
- # Inference
92
- descriptor = list(model.predict({'images': data}).values())[0] # [1, 8448]
93
- "
94
- ```
95
-
96
- ### Torch Hub (original float32)
97
-
98
- ```python
99
- import torch
100
- model = torch.hub.load("gmberton/MegaLoc", "get_trained_model")
101
- model.eval()
102
- with torch.no_grad():
103
- descriptor = model(image_tensor) # [B, 3, 518, 518] → [B, 8448]
104
- ```
105
-
106
- ## Quantization Details
107
-
108
- ### ONNX Models
109
-
110
- - **FP16**: Converted via `onnxconverter_common.float16.convert_float_to_float16()` with `keep_io_types=True` (input/output remain float32).
111
- - **INT8**: Per-channel symmetric weight quantization. Each weight tensor is independently quantized: `q = clamp(round(W / scale), -128, 127)` with `scale = max(|W|) / 127` per output channel. Weights stored as INT8 with `DequantizeLinear` nodes inserted in the graph.
112
-
113
- ### CoreML Models
114
-
115
- - **FP32/INT8**: Converted from PyTorch via `torch.export` + `coremltools.convert()` at macOS 15 deployment target.
116
- - **INT8**: Applied `linear_quantize_weights()` with symmetric per-channel INT8 linear quantization. No calibration data needed for weight-only quantization.
117
- - CoreML handles integer compute scheduling automatically — INT8 weights are dequantized on-the-fly inside GPU compute units with no extra latency.
118
-
119
- ## Reproducing
120
-
121
- ```bash
122
- # Clone and install dependencies
123
- git clone https://github.com/gmberton/MegaLoc
124
- cd MegaLoc
125
- pip install uv && uv sync
126
-
127
- # ONNX INT8 quantization
128
- uv run python quantize_megaloc.py --method 8bit --input megaloc.onnx
129
-
130
- # PyTorch → CoreML conversion + compression
131
- uv run python convert_coreml.py
132
-
133
- # Benchmark accuracy & speed (requires calibration images)
134
- uv run python benchmark_quantized.py --data-dir /path/to/images --runs 30
135
- ```
136
-
137
- ## Reference
138
-
139
- ```bibtex
140
- @inproceedings{Berton_2025_MegaLoc,
141
- author = {Berton, Gabriele and Masone, Carlo},
142
- title = {MegaLoc: One Retrieval to Place Them All},
143
- booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
144
- month = {June},
145
- year = {2025},
146
- pages = {2886--2892}
147
- }
148
- ```