ttw116 commited on
Commit
71af2e1
Β·
verified Β·
1 Parent(s): e777b40

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +206 -0
README.md CHANGED
@@ -1,3 +1,209 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ pipeline_tag: image-segmentation
4
+ tags:
5
+ - infrared-small-target-detection
6
+ - multimodal
7
+ - vision-language
8
+ - pytorch
9
+
10
  ---
11
+
12
+
13
+ <h1>ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection</h1>
14
+
15
+ <p>
16
+ <b>Tongtong Wang</b><sup>1</sup>&nbsp;
17
+ <b>Mingzhu Xu</b><sup>1βœ‰</sup>&nbsp;
18
+ <b>Chenglong Yu</b><sup>1</sup>&nbsp;
19
+ <b>Jing Wang</b><sup>1</sup>&nbsp;
20
+ <b>Xiaohui Lin</b><sup>1</sup>&nbsp;
21
+ <b>Weili Guan</b><sup>2</sup>
22
+ </p>
23
+
24
+
25
+ <p>
26
+ <sup>1</sup>School of Software, Shandong University<br>
27
+ <sup>2</sup>Harbin Institute of Technology, Shenzhen<br>
28
+ <sup>βœ‰</sup>Corresponding author
29
+ </p>
30
+
31
+
32
+ <p>
33
+ <a href="<paper-link>">
34
+ <img src="https://img.shields.io/badge/ACM%20MM-2026-blue" alt="ACM MM 2026">
35
+ </a>
36
+ <a href="https://github.com/iLearn-Lab/MM26-ADGNet">
37
+ <img src="https://img.shields.io/badge/GitHub-MM26--ADGNet-black?logo=github" alt="GitHub">
38
+ </a>
39
+ </p>
40
+
41
+ ---
42
+
43
+ ## πŸ“– Model Description
44
+
45
+ This repository provides the official model checkpoints for **ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection**, accepted by **ACM Multimedia 2026**.
46
+
47
+ Infrared Small Target Detection aims to accurately segment weak and tiny targets from complex infrared backgrounds. Existing pure-vision methods rely mainly on pixel-level information, while existing vision-language methods commonly describe targets and backgrounds using a single textual prompt. Such a symmetric design overlooks the inherent semantic differences between sparse infrared targets and structurally complex backgrounds.
48
+
49
+ ADGNet addresses this problem through three main components:
50
+
51
+ - **Asymmetric Dual-text Prompt (ADP):** uses an abstract, image-independent target prompt and a detailed, image-dependent background prompt.
52
+ - **Asymmetric Dual-Branch Interaction (ADBI):** independently performs target localization and background suppression using their corresponding textual priors.
53
+ - **Adaptive Feature Aggregation (AFA):** dynamically fuses target-enhanced and background-suppressed features for accurate segmentation.
54
+
55
+ The model uses the pretrained **CLIP ViT-B/16** text encoder to extract semantic representations from the target and background prompts.
56
+
57
+ ---
58
+
59
+ ## πŸ† Available Checkpoints
60
+
61
+ All ADGNet checkpoints are hosted in this Hugging Face model repository.
62
+
63
+ Download the required checkpoint directly from the **Files and versions** section of this repository.
64
+
65
+ | Dataset | Checkpoint |
66
+ | :--------: | :----------------------------------------------------------: |
67
+ | IRSTD-1K | [`ADGNet_mIoU_72.38_IRSTD-1K.pth.tar`](<irstd-1k-checkpoint-link>) |
68
+ | NUDT-SIRST | [`ADGNet_mIoU_95.53_NUDT-SIRST.pth.tar`](<nudt-sirst-checkpoint-link>) |
69
+ | SIRST | [`ADGNet_mIoU_83.08_SIRST.pth.tar`](<sirst-checkpoint-link>) |
70
+
71
+ ---
72
+
73
+ ## πŸš€ Usage
74
+
75
+ These checkpoints are designed to be used with the official ADGNet implementation:
76
+
77
+ ```text
78
+ https://github.com/iLearn-Lab/MM26-ADGNet
79
+ ```
80
+
81
+ ### 1. Clone the Official Repository
82
+
83
+ ```bash
84
+ git clone https://github.com/iLearn-Lab/MM26-ADGNet.git
85
+ cd MM26-ADGNet
86
+ ```
87
+
88
+ ### 2. Prepare the Checkpoints
89
+
90
+ Place the downloaded checkpoints in:
91
+
92
+ ```text
93
+ MM26-ADGNet/
94
+ └── SOTA_pth/
95
+ β”œβ”€β”€ ADGNet_mIoU_72.38_IRSTD-1K.pth.tar
96
+ β”œβ”€β”€ ADGNet_mIoU_95.53_NUDT-SIRST.pth.tar
97
+ └── ADGNet_mIoU_83.08_SIRST.pth.tar
98
+ ```
99
+
100
+ ### 3. Prepare the CLIP Text Encoder
101
+
102
+ ADGNet uses the pretrained **CLIP ViT-B/16** model:
103
+
104
+ ```bash
105
+ git clone https://huggingface.co/openai/clip-vit-base-patch16
106
+ ```
107
+
108
+ Update the local CLIP model path in the corresponding project configuration or source file before inference.
109
+
110
+ ### 4. Run Evaluation
111
+
112
+ Example evaluation on IRSTD-1K:
113
+
114
+ ```bash
115
+ python train.py \
116
+ --trainset "IRSTD-1K" \
117
+ --testset "IRSTD-1K" \
118
+ --dataset_dir "./datasets" \
119
+ --mode test \
120
+ --ckpt "./SOTA_pth/ADGNet_mIoU_72.38_IRSTD-1K.pth.tar"
121
+ ```
122
+
123
+ Replace the dataset name and checkpoint path when evaluating on NUDT-SIRST or SIRST.
124
+
125
+ ---
126
+
127
+ ## πŸ“‚ Dataset and Text Annotation Preparation
128
+
129
+ The original infrared images and ground-truth masks are not included in this model repository. Please obtain **IRSTD-1K**, **NUDT-SIRST**, and **SIRST** from their respective official sources.
130
+
131
+ The asymmetric text annotations used by ADGNet are released separately in our Hugging Face dataset repository:
132
+
133
+ - **AITIR Text Annotations:** [`Download`](<huggingface-text-dataset-link>)
134
+
135
+ After downloading the original datasets and text annotations, organize them according to the official ADGNet repository:
136
+
137
+ ```text
138
+ datasets/
139
+ β”œβ”€β”€ IRSTD-1K/
140
+ β”‚ β”œβ”€β”€ images/
141
+ β”‚ β”œβ”€β”€ masks/
142
+ β”‚ β”œβ”€β”€ img_idx/
143
+ β”‚ └── text/
144
+ β”œβ”€β”€ NUDT-SIRST/
145
+ β”‚ β”œβ”€β”€ images/
146
+ β”‚ β”œβ”€β”€ masks/
147
+ β”‚ β”œβ”€β”€ img_idx/
148
+ β”‚ └── text/
149
+ └── SIRST/
150
+ β”œβ”€β”€ images/
151
+ β”œβ”€β”€ masks/
152
+ β”œβ”€β”€ img_idx/
153
+ └── text/
154
+ ```
155
+
156
+ ---
157
+
158
+ ## 🎯 Intended Use
159
+
160
+ The released checkpoints are intended for:
161
+
162
+ - Academic research on infrared small target detection
163
+ - Reproduction of the results reported in the ADGNet paper
164
+ - Evaluation on IRSTD-1K, NUDT-SIRST, and SIRST
165
+ - Research on multimodal and text-guided infrared image segmentation
166
+ - Comparison with other infrared small target detection methods
167
+
168
+ ---
169
+
170
+ ## ⚠️ Limitations
171
+
172
+ - The model requires both infrared images and corresponding textual prompts.
173
+ - Detection performance may vary when applied to datasets or scenes that differ substantially from the training distribution.
174
+ - The released checkpoints are designed for the dataset splits and evaluation settings used in the paper.
175
+ - The model depends on the pretrained CLIP ViT-B/16 text encoder.
176
+ - The original infrared datasets are subject to their respective licenses and terms of use.
177
+
178
+ ---
179
+
180
+ ## πŸ”— Related Resources
181
+
182
+ - **GitHub Repository:** [iLearn-Lab/MM26-ADGNet](https://github.com/iLearn-Lab/MM26-ADGNet)
183
+ - **Paper:** [`ADGNet: Asymmetric Dual-text Guided Network for Infrared Small Target Detection`](<paper-link>)
184
+ - **Text Annotations:** [`AITIR Text Annotations`](<huggingface-text-dataset-link>)
185
+
186
+ ---
187
+
188
+ ## πŸ“š Citation
189
+
190
+ If you find ADGNet or the released checkpoints useful in your research, please consider citing our paper:
191
+
192
+ ```bibtex
193
+
194
+ ```
195
+
196
+ Please also consider checking out and citing our other related work:
197
+
198
+ ```bibtex
199
+
200
+ ```
201
+
202
+ ---
203
+
204
+ ## πŸ“„ License
205
+
206
+ This model repository is released under the [Apache License 2.0](./LICENSE).
207
+
208
+ The original datasets and the pretrained CLIP model remain subject to their respective licenses and terms of use.
209
+