insailab commited on
Commit
8801af1
Β·
verified Β·
1 Parent(s): d9d2e34

docs: add comprehensive Model Card for MatchFormer (ACCV 2022)

Browse files
Files changed (1) hide show
  1. README.md +105 -0
README.md ADDED
@@ -0,0 +1,105 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: feature-matching
3
+ library_name: pytorch
4
+ license: apache-2.0
5
+ tags:
6
+ - transformer
7
+ - feature-matching
8
+ - accv-2022
9
+ - vision
10
+ - scannet
11
+ - megadepth
12
+ datasets:
13
+ - scannet
14
+ - megadepth
15
+ ---
16
+
17
+ # MatchFormer: Interleaving Attention in Transformers for Feature Matching (ACCV 2022)
18
+
19
+ Official model checkpoints for **MatchFormer: Interleaving Attention in Transformers for Feature Matching** (ACCV 2022 Oral).
20
+
21
+ <div align="center">
22
+
23
+ [![Paper](https://img.shields.io/badge/arXiv-2203.09645-B31B1B.svg)](https://arxiv.org/abs/2203.09645)
24
+ [![GitHub](https://img.shields.io/badge/GitHub-MatchFormer-181717.svg?logo=github)](https://github.com/InSAI-Lab/MatchFormer)
25
+ [![Collection](https://img.shields.io/badge/HF-Collection-FFD21E.svg)](https://huggingface.co/collections/InSAI-Lab)
26
+
27
+ </div>
28
+
29
+ ---
30
+
31
+ ## πŸ“Œ Introduction
32
+
33
+ In this work, we propose a novel hierarchical extract-and-match transformer, termed **MatchFormer**. Inside each stage of the hierarchical encoder, we interleave self-attention for feature extraction and cross-attention for feature matching, enabling an intuitive and powerful **extract-and-match** scheme. MatchFormer establishes robust and accurate dense correspondences across diverse indoor and outdoor challenging scenes.
34
+
35
+ ![MatchFormer Architecture](https://raw.githubusercontent.com/InSAI-Lab/MatchFormer/main/matchformer.png)
36
+
37
+ ---
38
+
39
+ ## πŸ“‚ Model Zoo & File Structure
40
+
41
+ ```text
42
+ InSAI-Lab/MatchFormer
43
+ β”œβ”€β”€ indoor-large-SEA.ckpt # Indoor large model with SEA (Spatial-Entropy Attention) (89.5 MB)
44
+ β”œβ”€β”€ indoor-lite-LA.ckpt # Indoor lightweight model with LA (Linear Attention) (77.4 MB)
45
+ β”œβ”€β”€ outdoor-large-LA.ckpt # Outdoor large model with LA (77.4 MB)
46
+ └── outdoor-lite-SEA.ckpt # Outdoor lightweight model with SEA (103.5 MB)
47
+ ```
48
+
49
+ ---
50
+
51
+ ## πŸ“Š Benchmark Results
52
+
53
+ ### 1. Indoor Relative Pose Estimation on ScanNet (AUC)
54
+
55
+ | Model | Backbone | Resolution | AUC @ 5Β° | AUC @ 10Β° | AUC @ 20Β° | Weights |
56
+ | :--- | :--- | :---: | :---: | :---: | :---: | :--- |
57
+ | **MatchFormer-Large** | largesea | (8, 2) | **43.3** | **63.4** | **77.1** | [`indoor-large-SEA.ckpt`](https://huggingface.co/InSAI-Lab/MatchFormer/resolve/main/indoor-large-SEA.ckpt) |
58
+ | **MatchFormer-Lite** | litela | (8, 4) | **38.8** | **59.2** | **74.0** | [`indoor-lite-LA.ckpt`](https://huggingface.co/InSAI-Lab/MatchFormer/resolve/main/indoor-lite-LA.ckpt) |
59
+
60
+ ### 2. Outdoor Relative Pose Estimation on MegaDepth (AUC)
61
+
62
+ | Model | Backbone | Resolution | AUC @ 5Β° | AUC @ 10Β° | AUC @ 20Β° | Weights |
63
+ | :--- | :--- | :---: | :---: | :---: | :---: | :--- |
64
+ | **MatchFormer-Large** | largela | (8, 2) | **53.3** | **69.7** | **81.8** | [`outdoor-large-LA.ckpt`](https://huggingface.co/InSAI-Lab/MatchFormer/resolve/main/outdoor-large-LA.ckpt) |
65
+ | **MatchFormer-Lite** | litesea | (8, 4) | **48.2** | **65.1** | **78.4** | [`outdoor-lite-SEA.ckpt`](https://huggingface.co/InSAI-Lab/MatchFormer/resolve/main/outdoor-lite-SEA.ckpt) |
66
+
67
+ ---
68
+
69
+ ## ⚑ Quick Download Guide
70
+
71
+ ### Option 1: Hugging Face CLI
72
+
73
+ ```bash
74
+ # Download all MatchFormer weights to model/weights/
75
+ hf download InSAI-Lab/MatchFormer --local-dir model/weights
76
+
77
+ # Or download a specific checkpoint
78
+ hf download InSAI-Lab/MatchFormer indoor-large-SEA.ckpt --local-dir model/weights
79
+ ```
80
+
81
+ ### Option 2: Python `huggingface_hub`
82
+
83
+ ```python
84
+ from huggingface_hub import hf_hub_download
85
+
86
+ # Download indoor large SEA checkpoint
87
+ ckpt_path = hf_hub_download(
88
+ repo_id="InSAI-Lab/MatchFormer",
89
+ filename="indoor-large-SEA.ckpt",
90
+ local_dir="model/weights"
91
+ )
92
+ ```
93
+
94
+ ---
95
+
96
+ ## πŸ“– Citation
97
+
98
+ ```bibtex
99
+ @inproceedings{wang2022matchformer,
100
+ title={MatchFormer: Interleaving Attention in Transformers for Feature Matching},
101
+ author={Wang, Qing and Zhang, Jiaming and Yang, Kailun and Peng, Kunyu and Stiefelhagen, Rainer},
102
+ booktitle={Asian Conference on Computer Vision (ACCV)},
103
+ year={2022}
104
+ }
105
+ ```