Add pipeline tag, paper link, and GitHub link to model card

#8
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +59 -3
README.md CHANGED
@@ -1,3 +1,59 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ pipeline_tag: image-feature-extraction
4
+ ---
5
+
6
+ # CoTAP: Semantic Concentration for Self-Supervised Dense Representations Learning
7
+
8
+ This repository contains the official implementation for the paper "[Semantic Concentration for Self-Supervised Dense Representations Learning](https://huggingface.co/papers/2509.09429)", accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI 2025).
9
+
10
+ The code is available at: [https://github.com/KID-7391/CoTAP](https://github.com/KID-7391/CoTAP)
11
+
12
+ ## Overview
13
+
14
+ CoTAP addresses a key challenge in self-supervised learning: learning dense representations for image patches without the "over-dispersion" phenomenon, where patches from the same instance or category scatter, hindering performance on dense downstream tasks. The method proposes **explicit semantic concentration** through two main components:
15
+
16
+ 1. **Distilling Patch Correspondences**: To break strict spatial alignment, CoTAP distills patch correspondences, coupled with a novel noise-tolerant ranking loss (an extension of Average Precision loss to continuous targets) to robustly handle noisy and imbalanced pseudo-labels.
17
+ 2. **Object-Aware Filtering**: To discriminate shared patterns from complex scenes, an object-aware filter maps the output space to an object-based space, where patches are represented by learnable prototypes of objects via cross-attention.
18
+
19
+ This approach effectively learns semantically concentrated dense representations, demonstrating strong empirical results across various tasks.
20
+
21
+ ![CoTAP Pipeline](https://github.com/KID-7391/CoTAP/raw/main/overview-v2.png)
22
+
23
+ ## Checkpoints
24
+
25
+ | Method | Backbone | Linear Seg (mIoU) | kNN Cls (Acc) | Linear Cls (Acc) | Download |
26
+ | ------ | -------- | ----------------- | ------------- | ---------------- | -------- |
27
+ | DINO+CoTAP | VIT-S/16 | 50.5 | 78.0 | 79.1 | [link](https://huggingface.co/KID-7391/CoTAP/blob/main/checkpoints/ours/vits16/dino%2Bcotap.ckpt) |
28
+ | Leopart+CoTAP | VIT-S/16 | 50.4 | 77.3 | 78.6 | [link](https://huggingface.co/KID-7391/CoTAP/blob/main/checkpoints/ours/vits16/leopart%2Bcotap.ckpt) |
29
+ | iBOT+CoTAP | VIT-S/16 | 51.6 | 78.4 | 79.4 | [link](https://huggingface.co/KID-7391/CoTAP/blob/main/checkpoints/ours/vits16/ibot%2Bcotap.ckpt) |
30
+ | Mugs+CoTAP | VIT-S/16 | 51.6 | 78.9 | 79.4 | [link](https://huggingface.co/KID-7391/CoTAP/blob/main/checkpoints/ours/vits16/mugs%2Bcotap.ckpt) |
31
+ | DINO+CoTAP | VIT-B/16 | 52.2 | 79.2 | 80.2 | [link](https://huggingface.co/KID-7391/CoTAP/blob/main/checkpoints/ours/vitb16/dino%2Bcotap.ckpt) |
32
+ | iBOT+CoTAP | VIT-B/16 | 53.9 | 80.1 | 80.8 | [link](https://huggingface.co/KID-7391/CoTAP/blob/main/checkpoints/ours/vitb16/ibot%2Bcotap.ckpt) |
33
+ | Mugs+CoTAP | VIT-B/16 | 52.1 | 80.8 | 81.7 | [link](https://huggingface.co/KID-7391/CoTAP/blob/main/checkpoints/ours/vitb16/mugs%2Bcotap.ckpt) |
34
+ | DINO+CoTAP | VIT-B/8 | 54.7 | 81.6 | 82.1 | [link](https://huggingface.co/KID-7391/CoTAP/blob/main/checkpoints/ours/vitb8/dino%2Bcotap.ckpt) |
35
+
36
+ ## Installation and Usage
37
+
38
+ For detailed instructions on environment setup, installation, dataset preparation, training, and evaluation, please refer to the [official GitHub repository](https://github.com/KID-7391/CoTAP).
39
+
40
+ ## Citation
41
+
42
+ If you find this work useful in your research, please cite the following paper:
43
+
44
+ ```bibtex
45
+ @article{wen2025semantic,
46
+ title={Semantic Concentration for Self-Supervised Dense Representations Learning},
47
+ author={Wen, Peisong and Xu, Qianqian and Dai, Siran and Cong, Runmin and Huang, Qingming},
48
+ journal={IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
49
+ year={2025},
50
+ eprint={2509.09429},
51
+ archivePrefix={arXiv},
52
+ primaryClass={cs.CV},
53
+ url={https://arxiv.org/abs/2509.09429},
54
+ }
55
+ ```
56
+
57
+ ## Acknowledgements
58
+
59
+ Our code is based on the official PyTorch implementation of [DINO](https://github.com/facebookresearch/dino), [Leopart](https://github.com/MkuuWaUjinga/leopart), and [STEGO](https://github.com/mhamilton723/STEGO).