Fix placeholders and add citation
Browse filesThis PR fixes the placeholders in the model card, adds links to the paper page and the associated dataset repository, and completes the citation entry so users can easily reference the work.
README.md
CHANGED
|
@@ -1,197 +1,216 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
pipeline_tag: image-segmentation
|
| 4 |
-
tags:
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
---
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
<
|
| 15 |
-
<b>
|
| 16 |
-
<b>
|
| 17 |
-
<b>
|
| 18 |
-
<b>
|
| 19 |
-
<b>
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
<
|
| 24 |
-
<sup>
|
| 25 |
-
<sup>
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
<
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
<
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
- **Asymmetric Dual-
|
| 47 |
-
- **
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
|
| 61 |
-
|
|
| 62 |
-
|
|
| 63 |
-
|
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
├──
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
--
|
| 112 |
-
--
|
| 113 |
-
--
|
| 114 |
-
--
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
|
| 134 |
-
├──
|
| 135 |
-
│ ├──
|
| 136 |
-
│ ├──
|
| 137 |
-
│
|
| 138 |
-
|
| 139 |
-
├──
|
| 140 |
-
│ ├──
|
| 141 |
-
│ ├──
|
| 142 |
-
│
|
| 143 |
-
|
| 144 |
-
|
| 145 |
-
├──
|
| 146 |
-
├──
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
-
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
|
| 156 |
-
|
| 157 |
-
-
|
| 158 |
-
-
|
| 159 |
-
-
|
| 160 |
-
-
|
| 161 |
-
|
| 162 |
-
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
-
|
| 168 |
-
-
|
| 169 |
-
- The
|
| 170 |
-
- The
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
- **
|
| 178 |
-
- **
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
|
| 188 |
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
```
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
pipeline_tag: image-segmentation
|
| 4 |
+
tags:
|
| 5 |
+
- infrared-small-target-detection
|
| 6 |
+
- multimodal
|
| 7 |
+
- vision-language
|
| 8 |
+
- pytorch
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
<h1>ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection</h1>
|
| 12 |
+
|
| 13 |
+
<p>
|
| 14 |
+
<b>Tongtong Wang</b><sup>1</sup>
|
| 15 |
+
<b>Mingzhu Xu</b><sup>1✉</sup>
|
| 16 |
+
<b>Chenglong Yu</b><sup>1</sup>
|
| 17 |
+
<b>Jing Wang</b><sup>1</sup>
|
| 18 |
+
<b>Xiaohui Lin</b><sup>1</sup>
|
| 19 |
+
<b>Weili Guan</b><sup>2</sup>
|
| 20 |
+
</p>
|
| 21 |
+
|
| 22 |
+
<p>
|
| 23 |
+
<sup>1</sup>School of Software, Shandong University<br>
|
| 24 |
+
<sup>2</sup>Harbin Institute of Technology, Shenzhen<br>
|
| 25 |
+
<sup>✉</sup>Corresponding author
|
| 26 |
+
</p>
|
| 27 |
+
|
| 28 |
+
<p>
|
| 29 |
+
<a href="https://huggingface.co/papers/2609.00853">
|
| 30 |
+
<img src="https://img.shields.io/badge/ACM%20MM-2026-blue" alt="ACM MM 2026">
|
| 31 |
+
</a>
|
| 32 |
+
<a href="https://github.com/iLearn-Lab/MM26-ADGNet">
|
| 33 |
+
<img src="https://img.shields.io/badge/GitHub-MM26--ADGNet-black?logo=github" alt="GitHub">
|
| 34 |
+
</a>
|
| 35 |
+
</p>
|
| 36 |
+
|
| 37 |
+
## 📖 Model Description
|
| 38 |
+
|
| 39 |
+
This repository provides the official model checkpoints for **ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection**, accepted by **ACM Multimedia 2026**.
|
| 40 |
+
|
| 41 |
+
Infrared Small Target Detection aims to accurately segment weak and tiny targets from complex infrared backgrounds. Existing pure-vision methods rely mainly on pixel-level information, while existing vision-language methods commonly describe targets and backgrounds using a single textual prompt. Such a symmetric design overlooks the inherent semantic differences between sparse infrared targets and structurally complex backgrounds.
|
| 42 |
+
|
| 43 |
+
ADGNet addresses this problem through three main components:
|
| 44 |
+
|
| 45 |
+
- **Asymmetric Dual-text Prompt (ADP):** uses an abstract, image-independent target prompt and a detailed, image-dependent background prompt.
|
| 46 |
+
- **Asymmetric Dual-Branch Interaction (ADBI):** independently performs target localization and background suppression using their corresponding textual priors.
|
| 47 |
+
- **Adaptive Feature Aggregation (AFA):** dynamically fuses target-enhanced and background-suppressed features for accurate segmentation.
|
| 48 |
+
|
| 49 |
+
The model uses the pretrained **CLIP ViT-B/16** text encoder to extract semantic representations from the target and background prompts.
|
| 50 |
+
|
| 51 |
+
---
|
| 52 |
+
|
| 53 |
+
## 🏆 Available Checkpoints
|
| 54 |
+
|
| 55 |
+
All ADGNet checkpoints are hosted in this Hugging Face model repository.
|
| 56 |
+
|
| 57 |
+
Download the required checkpoint directly from the **Files and versions** section of this repository.
|
| 58 |
+
|
| 59 |
+
| Dataset | Checkpoint |
|
| 60 |
+
| :--------: | :----------------------------------------------------------: |
|
| 61 |
+
| IRSTD-1K | [`ADGNet_mIoU_72.38_IRSTD-1K.pth.tar`](https://huggingface.co/iLearn-Lab/MM26-ADGNet/blob/main/ADGNet_mIoU_72.38_IRSTD-1K.pth.tar) |
|
| 62 |
+
| NUDT-SIRST | [`ADGNet_mIoU_95.53_NUDT-SIRST.pth.tar`](https://huggingface.co/iLearn-Lab/MM26-ADGNet/blob/main/ADGNet_mIoU_95.53_NUDT-SIRST.pth.tar) |
|
| 63 |
+
| SIRST | [`ADGNet_mIoU_83.08_SIRST.pth.tar`](https://huggingface.co/iLearn-Lab/MM26-ADGNet/blob/main/ADGNet_mIoU_83.08_SIRST.pth.tar) |
|
| 64 |
+
|
| 65 |
+
---
|
| 66 |
+
|
| 67 |
+
## 🚀 Usage
|
| 68 |
+
|
| 69 |
+
These checkpoints are designed to be used with the official ADGNet implementation:
|
| 70 |
+
|
| 71 |
+
```text
|
| 72 |
+
https://github.com/iLearn-Lab/MM26-ADGNet
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
### 1. Clone the Official Repository
|
| 76 |
+
|
| 77 |
+
```bash
|
| 78 |
+
git clone https://github.com/iLearn-Lab/MM26-ADGNet.git
|
| 79 |
+
cd MM26-ADGNet
|
| 80 |
+
```
|
| 81 |
+
|
| 82 |
+
### 2. Prepare the Checkpoints
|
| 83 |
+
|
| 84 |
+
Place the downloaded checkpoints in:
|
| 85 |
+
|
| 86 |
+
```text
|
| 87 |
+
MM26-ADGNet/
|
| 88 |
+
└── SOTA_pth/
|
| 89 |
+
├── ADGNet_mIoU_72.38_IRSTD-1K.pth.tar
|
| 90 |
+
├── ADGNet_mIoU_95.53_NUDT-SIRST.pth.tar
|
| 91 |
+
└── ADGNet_mIoU_83.08_SIRST.pth.tar
|
| 92 |
+
```
|
| 93 |
+
|
| 94 |
+
### 3. Prepare the CLIP Text Encoder
|
| 95 |
+
|
| 96 |
+
ADGNet uses the pretrained **CLIP ViT-B/16** model:
|
| 97 |
+
|
| 98 |
+
```bash
|
| 99 |
+
git clone https://huggingface.co/openai/clip-vit-base-patch16
|
| 100 |
+
```
|
| 101 |
+
|
| 102 |
+
Update the local CLIP model path in the corresponding project configuration or source file before inference.
|
| 103 |
+
|
| 104 |
+
### 4. Run Evaluation
|
| 105 |
+
|
| 106 |
+
Example evaluation on IRSTD-1K:
|
| 107 |
+
|
| 108 |
+
```bash
|
| 109 |
+
python train.py \
|
| 110 |
+
--trainset "IRSTD-1K" \
|
| 111 |
+
--testset "IRSTD-1K" \
|
| 112 |
+
--dataset_dir "./datasets" \
|
| 113 |
+
--mode test \
|
| 114 |
+
--ckpt "./SOTA_pth/ADGNet_mIoU_72.38_IRSTD-1K.pth.tar"
|
| 115 |
+
```
|
| 116 |
+
|
| 117 |
+
Replace the dataset name and checkpoint path when evaluating on NUDT-SIRST or SIRST.
|
| 118 |
+
|
| 119 |
+
---
|
| 120 |
+
|
| 121 |
+
## 📂 Dataset and Text Annotation Preparation
|
| 122 |
+
|
| 123 |
+
The original infrared images and ground-truth masks are not included in this model repository. Please obtain **IRSTD-1K**, **NUDT-SIRST**, and **SIRST** from their respective official sources.
|
| 124 |
+
|
| 125 |
+
The asymmetric text annotations used by ADGNet are released separately in our Hugging Face dataset repository:
|
| 126 |
+
|
| 127 |
+
- **AITIR Text Annotations:** [`Download`](https://huggingface.co/datasets/iLearn-Lab/MM26-ADGNet-AITIR-Text)
|
| 128 |
+
|
| 129 |
+
After downloading the original datasets and text annotations, organize them according to the official ADGNet repository:
|
| 130 |
+
|
| 131 |
+
```text
|
| 132 |
+
datasets/
|
| 133 |
+
├── IRSTD-1K/
|
| 134 |
+
│ ├── images/
|
| 135 |
+
│ ├── masks/
|
| 136 |
+
│ ├── img_idx/
|
| 137 |
+
│ └── text/
|
| 138 |
+
├── NUDT-SIRST/
|
| 139 |
+
│ ├── images/
|
| 140 |
+
│ ├── masks/
|
| 141 |
+
│ ├── img_idx/
|
| 142 |
+
│ └── text/
|
| 143 |
+
└── SIRST/
|
| 144 |
+
├── images/
|
| 145 |
+
├── masks/
|
| 146 |
+
├── img_idx/
|
| 147 |
+
└── text/
|
| 148 |
+
```
|
| 149 |
+
|
| 150 |
+
---
|
| 151 |
+
|
| 152 |
+
## 🎯 Intended Use
|
| 153 |
+
|
| 154 |
+
The released checkpoints are intended for:
|
| 155 |
+
|
| 156 |
+
- Academic research on infrared small target detection
|
| 157 |
+
- Reproduction of the results reported in the ADGNet paper
|
| 158 |
+
- Evaluation on IRSTD-1K, NUDT-SIRST, and SIRST
|
| 159 |
+
- Research on multimodal and text-guided infrared image segmentation
|
| 160 |
+
- Comparison with other infrared small target detection methods
|
| 161 |
+
|
| 162 |
+
---
|
| 163 |
+
|
| 164 |
+
## ⚠️ Limitations
|
| 165 |
+
|
| 166 |
+
- The model requires both infrared images and corresponding textual prompts.
|
| 167 |
+
- Detection performance may vary when applied to datasets or scenes that differ substantially from the training distribution.
|
| 168 |
+
- The released checkpoints are designed for the dataset splits and evaluation settings used in the paper.
|
| 169 |
+
- The model depends on the pretrained CLIP ViT-B/16 text encoder.
|
| 170 |
+
- The original infrared datasets are subject to their respective licenses and terms of use.
|
| 171 |
+
|
| 172 |
+
---
|
| 173 |
+
|
| 174 |
+
## 🔗 Related Resources
|
| 175 |
+
|
| 176 |
+
- **GitHub Repository:** [iLearn-Lab/MM26-ADGNet](https://github.com/iLearn-Lab/MM26-ADGNet)
|
| 177 |
+
- **Paper:** [`ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection`](https://huggingface.co/papers/2609.00853)
|
| 178 |
+
- **Text Annotations:** [`AITIR Text Annotations`](https://huggingface.co/datasets/iLearn-Lab/MM26-ADGNet-AITIR-Text)
|
| 179 |
+
|
| 180 |
+
---
|
| 181 |
+
|
| 182 |
+
## 📚 Citation
|
| 183 |
+
|
| 184 |
+
If you find ADGNet or the released checkpoints useful in your research, please consider citing our paper:
|
| 185 |
+
|
| 186 |
+
```bibtex
|
| 187 |
+
@inproceedings{wang2026adgnet,
|
| 188 |
+
title = {ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection},
|
| 189 |
+
author = {Wang, Tongtong and Xu, Mingzhu and Yu, Chenglong and Wang, Jing and Lin, Xiaohui and Guan, Weili},
|
| 190 |
+
booktitle = {Proceedings of the ACM International Conference on Multimedia},
|
| 191 |
+
year = {2026},
|
| 192 |
+
}
|
| 193 |
+
```
|
| 194 |
+
|
| 195 |
+
Please also consider checking out and citing our other related work:
|
| 196 |
+
|
| 197 |
+
```bibtex
|
| 198 |
+
@inproceedings{yu2026dgnet,
|
| 199 |
+
title = {DGNet: Dual-knowledge Guided Network for Infrared Small Target Detection},
|
| 200 |
+
author = {Yu, Chenglong and Xu, Mingzhu and Wang, Jing and Wang, Tongtong and Miao, Pingping and Nie, Liqiang},
|
| 201 |
+
booktitle = {Proceedings of the ACM International Conference on Multimedia},
|
| 202 |
+
year = {2026},
|
| 203 |
+
}
|
| 204 |
+
|
| 205 |
+
@article{11017756,
|
| 206 |
+
author = {Xu, Mingzhu and Yu, Chenglong and Li, Zexuan and Tang, Haoyu and Hu, Yupeng and Nie, Liqiang},
|
| 207 |
+
journal = {IEEE Transactions on Geoscience and Remote Sensing},
|
| 208 |
+
title = {HDNet: A Hybrid Domain Network With Multiscale High-Frequency Information Enhancement for Infrared Small-Target Detection},
|
| 209 |
+
year = {2025},
|
| 210 |
+
volume = {63},
|
| 211 |
+
pages = {1--15},
|
| 212 |
+
doi = {10.1109/TGRS.2025.3574962},
|
| 213 |
+
}
|
| 214 |
+
```
|
| 215 |
+
|
| 216 |
+
---
|