Delete megaloc/README.md with huggingface_hub
Browse files- megaloc/README.md +0 -148
megaloc/README.md
DELETED
|
@@ -1,148 +0,0 @@
|
|
| 1 |
-
# MegaLoc Quantized Models
|
| 2 |
-
|
| 3 |
-
Quantized ONNX and CoreML models for [MegaLoc](https://arxiv.org/abs/2502.17237) — a state-of-the-art visual place recognition model based on DINOv2 ViT-B/14.
|
| 4 |
-
|
| 5 |
-
All benchmarks below were run on a MacBook Pro with Apple Silicon (M-series), 518×518 input, single-threaded CPU for ONNX Runtime, GPU for CoreML.
|
| 6 |
-
|
| 7 |
-
## Available Models
|
| 8 |
-
|
| 9 |
-
| File | Format | Size | Ratio | CosSim | Latency | FPS | Notes |
|
| 10 |
-
|------|--------|------|-------|--------|---------|-----|-------|
|
| 11 |
-
| `megaloc.onnx` | ONNX FP32 | 873 MB | 1.0× | 1.000000 | 387 ms | 2.6 | Baseline, float32 reference |
|
| 12 |
-
| `megaloc_fp16.onnx` | ONNX FP16 | 437 MB | 2.0× | 0.999988 | 394 ms | 2.5 | Near-lossless, recommended for ONNX |
|
| 13 |
-
| `megaloc_int8.onnx` | ONNX INT8 | 223 MB | 3.9× | 0.996453 | 475 ms | 2.1 | Weight-only quantization, Recall@1 still 100% |
|
| 14 |
-
| `megaloc_coreml_fp32.mlpackage` | CoreML FP32 | 873 MB | 1.0× | 1.000000 | 78 ms | 12.8 | CoreML baseline, 5× faster than ONNX |
|
| 15 |
-
| `megaloc_coreml_int8.mlpackage` | CoreML INT8 | 219 MB | 4.0× | 0.998315 | 78 ms | 12.9 | **Recommended** — best size/speed/accuracy trade-off |
|
| 16 |
-
|
| 17 |
-
## Benchmark Summary
|
| 18 |
-
|
| 19 |
-
```
|
| 20 |
-
Model Size Ratio CosSim Latency FPS
|
| 21 |
-
─────────────────────────────────────────────────────────────────────────
|
| 22 |
-
ONNX FP32 872.9 MB 1.0× 1.000000 387 ms 2.6
|
| 23 |
-
ONNX FP16 436.8 MB 2.0× 0.999988 394 ms 2.5
|
| 24 |
-
ONNX INT8 222.5 MB 3.9× 0.996453 475 ms 2.1
|
| 25 |
-
CoreML FP32 872.8 MB 1.0× 1.000000 78 ms 12.8
|
| 26 |
-
CoreML INT8 218.8 MB 4.0× 0.998315 78 ms 12.9
|
| 27 |
-
```
|
| 28 |
-
|
| 29 |
-
### Retrieval Accuracy
|
| 30 |
-
|
| 31 |
-
| Model | Recall@1 | Recall@3 | Recall@5 | Kendall Tau |
|
| 32 |
-
|-------|----------|----------|----------|-------------|
|
| 33 |
-
| ONNX FP16 | 100% | 100% | 100% | 0.94 |
|
| 34 |
-
| ONNX INT8 | 100% | 100% | 100% | 0.58 |
|
| 35 |
-
|
| 36 |
-
Despite a drop in descriptor-level cosine similarity, INT8 models preserve top-K retrieval perfectly. The full ranking order (Kendall Tau) shows some reshuffling, but the most relevant results remain correct.
|
| 37 |
-
|
| 38 |
-
### Why CoreML is faster than ONNX Runtime?
|
| 39 |
-
|
| 40 |
-
- ONNX Runtime uses CPU execution by default on macOS, while **CoreML leverages Apple Silicon GPU** via Metal Performance Shaders
|
| 41 |
-
- ONNX INT8 is slower (475 ms vs 387 ms) because the weight-only quantization inserts `DequantizeLinear` nodes — each MatMul first converts INT8 back to float32, adding overhead without the speed benefit of integer math
|
| 42 |
-
- CoreML INT8 runs at the same speed as FP32 (78 ms) because the GPU's integer compute units are used natively
|
| 43 |
-
|
| 44 |
-
### On sub-8-bit quantization
|
| 45 |
-
|
| 46 |
-
We experimented with 4-bit and 1-bit quantization (ONNX INT4, CoreML 4-bit/2-bit K-Means palettization). **All failed** with cosine similarity ≈ 0. The DINOv2 ViT backbone has 12 transformer blocks + aggregator (~200 weight tensors). Quantization error from only 16 levels accumulates catastrophically across 48+ MatMul layers. This is a known limitation — ViT architectures below 8-bit require **quantization-aware training (QAT)** to preserve accuracy.
|
| 47 |
-
|
| 48 |
-
## Usage
|
| 49 |
-
|
| 50 |
-
### ONNX Inference
|
| 51 |
-
|
| 52 |
-
```bash
|
| 53 |
-
pip install onnxruntime pillow numpy
|
| 54 |
-
|
| 55 |
-
python3 -c "
|
| 56 |
-
import onnxruntime as ort, numpy as np
|
| 57 |
-
from PIL import Image
|
| 58 |
-
|
| 59 |
-
# Choose from the table above
|
| 60 |
-
session = ort.InferenceSession('megaloc_fp16.onnx', providers=['CPUExecutionProvider'])
|
| 61 |
-
|
| 62 |
-
# Preprocess: resize to 518×518, normalize to [0, 1]
|
| 63 |
-
img = Image.open('image.jpg').convert('RGB').resize((518, 518), Image.LANCZOS)
|
| 64 |
-
data = np.array(img).astype(np.float32) / 255.0
|
| 65 |
-
data = np.expand_dims(data.transpose(2, 0, 1), axis=0) # [1, 3, 518, 518]
|
| 66 |
-
|
| 67 |
-
# Inference → L2-normalized descriptor [1, 8448]
|
| 68 |
-
descriptor = session.run(None, {'images': data})[0]
|
| 69 |
-
|
| 70 |
-
# Compare two images via cosine similarity
|
| 71 |
-
cos_sim = np.dot(desc1, desc2.T) # both already L2-normalized
|
| 72 |
-
"
|
| 73 |
-
```
|
| 74 |
-
|
| 75 |
-
### CoreML Inference (macOS only)
|
| 76 |
-
|
| 77 |
-
```bash
|
| 78 |
-
pip install coremltools pillow numpy
|
| 79 |
-
|
| 80 |
-
python3 -c "
|
| 81 |
-
import coremltools as ct, numpy as np
|
| 82 |
-
from PIL import Image
|
| 83 |
-
|
| 84 |
-
model = ct.models.MLModel('megaloc_coreml_int8.mlpackage')
|
| 85 |
-
|
| 86 |
-
# Preprocess
|
| 87 |
-
img = Image.open('image.jpg').convert('RGB').resize((518, 518), Image.LANCZOS)
|
| 88 |
-
data = np.array(img).astype(np.float32) / 255.0
|
| 89 |
-
data = np.expand_dims(data.transpose(2, 0, 1), axis=0)
|
| 90 |
-
|
| 91 |
-
# Inference
|
| 92 |
-
descriptor = list(model.predict({'images': data}).values())[0] # [1, 8448]
|
| 93 |
-
"
|
| 94 |
-
```
|
| 95 |
-
|
| 96 |
-
### Torch Hub (original float32)
|
| 97 |
-
|
| 98 |
-
```python
|
| 99 |
-
import torch
|
| 100 |
-
model = torch.hub.load("gmberton/MegaLoc", "get_trained_model")
|
| 101 |
-
model.eval()
|
| 102 |
-
with torch.no_grad():
|
| 103 |
-
descriptor = model(image_tensor) # [B, 3, 518, 518] → [B, 8448]
|
| 104 |
-
```
|
| 105 |
-
|
| 106 |
-
## Quantization Details
|
| 107 |
-
|
| 108 |
-
### ONNX Models
|
| 109 |
-
|
| 110 |
-
- **FP16**: Converted via `onnxconverter_common.float16.convert_float_to_float16()` with `keep_io_types=True` (input/output remain float32).
|
| 111 |
-
- **INT8**: Per-channel symmetric weight quantization. Each weight tensor is independently quantized: `q = clamp(round(W / scale), -128, 127)` with `scale = max(|W|) / 127` per output channel. Weights stored as INT8 with `DequantizeLinear` nodes inserted in the graph.
|
| 112 |
-
|
| 113 |
-
### CoreML Models
|
| 114 |
-
|
| 115 |
-
- **FP32/INT8**: Converted from PyTorch via `torch.export` + `coremltools.convert()` at macOS 15 deployment target.
|
| 116 |
-
- **INT8**: Applied `linear_quantize_weights()` with symmetric per-channel INT8 linear quantization. No calibration data needed for weight-only quantization.
|
| 117 |
-
- CoreML handles integer compute scheduling automatically — INT8 weights are dequantized on-the-fly inside GPU compute units with no extra latency.
|
| 118 |
-
|
| 119 |
-
## Reproducing
|
| 120 |
-
|
| 121 |
-
```bash
|
| 122 |
-
# Clone and install dependencies
|
| 123 |
-
git clone https://github.com/gmberton/MegaLoc
|
| 124 |
-
cd MegaLoc
|
| 125 |
-
pip install uv && uv sync
|
| 126 |
-
|
| 127 |
-
# ONNX INT8 quantization
|
| 128 |
-
uv run python quantize_megaloc.py --method 8bit --input megaloc.onnx
|
| 129 |
-
|
| 130 |
-
# PyTorch → CoreML conversion + compression
|
| 131 |
-
uv run python convert_coreml.py
|
| 132 |
-
|
| 133 |
-
# Benchmark accuracy & speed (requires calibration images)
|
| 134 |
-
uv run python benchmark_quantized.py --data-dir /path/to/images --runs 30
|
| 135 |
-
```
|
| 136 |
-
|
| 137 |
-
## Reference
|
| 138 |
-
|
| 139 |
-
```bibtex
|
| 140 |
-
@inproceedings{Berton_2025_MegaLoc,
|
| 141 |
-
author = {Berton, Gabriele and Masone, Carlo},
|
| 142 |
-
title = {MegaLoc: One Retrieval to Place Them All},
|
| 143 |
-
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
|
| 144 |
-
month = {June},
|
| 145 |
-
year = {2025},
|
| 146 |
-
pages = {2886--2892}
|
| 147 |
-
}
|
| 148 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|