metadata
pipeline_tag: feature-matching
library_name: pytorch
license: apache-2.0
tags:
- transformer
- feature-matching
- accv-2022
- vision
- scannet
- megadepth
datasets:
- scannet
- megadepth
MatchFormer: Interleaving Attention in Transformers for Feature Matching (ACCV 2022)
Official model checkpoints for MatchFormer: Interleaving Attention in Transformers for Feature Matching (ACCV 2022 Oral).
📌 Introduction
In this work, we propose a novel hierarchical extract-and-match transformer, termed MatchFormer. Inside each stage of the hierarchical encoder, we interleave self-attention for feature extraction and cross-attention for feature matching, enabling an intuitive and powerful extract-and-match scheme. MatchFormer establishes robust and accurate dense correspondences across diverse indoor and outdoor challenging scenes.
📂 Model Zoo & File Structure
InSAI-Lab/MatchFormer
├── indoor-large-SEA.ckpt # Indoor large model with SEA (Spatial-Entropy Attention) (89.5 MB)
├── indoor-lite-LA.ckpt # Indoor lightweight model with LA (Linear Attention) (77.4 MB)
├── outdoor-large-LA.ckpt # Outdoor large model with LA (77.4 MB)
└── outdoor-lite-SEA.ckpt # Outdoor lightweight model with SEA (103.5 MB)
📊 Benchmark Results
1. Indoor Relative Pose Estimation on ScanNet (AUC)
| Model | Backbone | Resolution | AUC @ 5° | AUC @ 10° | AUC @ 20° | Weights |
|---|---|---|---|---|---|---|
| MatchFormer-Large | largesea | (8, 2) | 43.3 | 63.4 | 77.1 | indoor-large-SEA.ckpt |
| MatchFormer-Lite | litela | (8, 4) | 38.8 | 59.2 | 74.0 | indoor-lite-LA.ckpt |
2. Outdoor Relative Pose Estimation on MegaDepth (AUC)
| Model | Backbone | Resolution | AUC @ 5° | AUC @ 10° | AUC @ 20° | Weights |
|---|---|---|---|---|---|---|
| MatchFormer-Large | largela | (8, 2) | 53.3 | 69.7 | 81.8 | outdoor-large-LA.ckpt |
| MatchFormer-Lite | litesea | (8, 4) | 48.2 | 65.1 | 78.4 | outdoor-lite-SEA.ckpt |
⚡ Quick Download Guide
Option 1: Hugging Face CLI
# Download all MatchFormer weights to model/weights/
hf download InSAI-Lab/MatchFormer --local-dir model/weights
# Or download a specific checkpoint
hf download InSAI-Lab/MatchFormer indoor-large-SEA.ckpt --local-dir model/weights
Option 2: Python huggingface_hub
from huggingface_hub import hf_hub_download
# Download indoor large SEA checkpoint
ckpt_path = hf_hub_download(
repo_id="InSAI-Lab/MatchFormer",
filename="indoor-large-SEA.ckpt",
local_dir="model/weights"
)
📖 Citation
@inproceedings{wang2022matchformer,
title={MatchFormer: Interleaving Attention in Transformers for Feature Matching},
author={Wang, Qing and Zhang, Jiaming and Yang, Kailun and Peng, Kunyu and Stiefelhagen, Rainer},
booktitle={Asian Conference on Computer Vision (ACCV)},
year={2022}
}