JunDong-dev commited on
Commit
bc15982
·
verified ·
1 Parent(s): ac02d47

Add model card

Browse files
Files changed (1) hide show
  1. README.md +96 -1
README.md CHANGED
@@ -1,3 +1,98 @@
1
  ---
2
- license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: agpl-3.0
3
+ library_name: pytorch
4
+ pipeline_tag: object-detection
5
+ tags:
6
+ - remote-sensing
7
+ - visual-grounding
8
+ - sar
9
+ - optical
10
+ - cross-domain
11
+ - mixture-of-experts
12
+ - contrastive-learning
13
+ - benchmark
14
  ---
15
+
16
+ # OptiSAR-Net++ — Official Weights
17
+
18
+ Official trained weights (`best.pt`) of **OptiSAR-Net++: A Large-Scale Benchmark and Transformer-Free Framework for Cross-Domain Remote Sensing Visual Grounding**.
19
+
20
+ OptiSAR-Net++ is a **transformer-free** framework for Cross-Domain Remote Sensing Visual Grounding (CD-RSVG): a single unified model localizes targets described by free-form natural language in **both optical and SAR** remote sensing imagery, replacing a heavy Transformer decoder with **contrastive region–text matching**.
21
+
22
+ - 🤗 **Dataset**: [JunDong-dev/OptSAR-RSVG](https://huggingface.co/datasets/JunDong-dev/OptSAR-RSVG)
23
+ - 💻 **Code**: [GitHub — JunDong-dev/OptiSAR-Net-PlusPlus](https://github.com/JunDong-dev/OptiSAR-Net-PlusPlus)
24
+
25
+ ## 🏗️ Architecture
26
+
27
+ The checkpoint follows a YOLOE-style single-stage detector extended with:
28
+
29
+ | Component | Location | Role |
30
+ |---|---|---|
31
+ | **PLoRA-MoE** | Backbone | Patch-level low-rank-adaptation Mixture-of-Experts for optical/SAR feature disentanglement |
32
+ | **TGDG-SSA** | Neck (×3) | Language-guided multi-scale fusion of visual features with text embeddings |
33
+ | **OptiSARNetPlusPlusDetect** | Head | Region–text contrastive matching head with region-aware auxiliary supervision |
34
+ | MobileCLIP2-B (frozen) | Text encoder | Encodes referring expressions |
35
+
36
+ Key config of this checkpoint: `nc: 16` classes, `scale: m`, `reg_max: 16`.
37
+
38
+ ## 📦 Files
39
+
40
+ | File | Description |
41
+ |---|---|
42
+ | `best.pt` | Trained weights (≈ 129 MB). The model architecture/modules are defined in the [GitHub repository](https://github.com/JunDong-dev/OptiSAR-Net-PlusPlus). |
43
+
44
+ ## 📈 Results — OptSAR-RSVG Test Split
45
+
46
+ Evaluated on the [OptSAR-RSVG](https://huggingface.co/datasets/JunDong-dev/OptSAR-RSVG) test split (4,434 images / 8,103 referring expressions). All values in %.
47
+
48
+ | Domain | Samples | Pr@0.5 | Pr@0.6 | Pr@0.7 | Pr@0.8 | Pr@0.9 | meanIoU | cumIoU |
49
+ |---|---:|---:|---:|---:|---:|---:|---:|---:|
50
+ | **All** | 8,103 | **93.61** | 93.19 | 91.94 | 87.75 | 66.43 | **85.94** | **92.11** |
51
+ | Optical | 6,027 | 93.01 | 92.58 | 91.67 | 88.58 | 72.99 | 86.48 | 92.25 |
52
+ | SAR | 2,076 | 95.33 | 94.94 | 92.73 | 85.31 | 47.40 | 84.38 | 83.21 |
53
+
54
+ Benchmark comparisons against TransVG, LQVG, TACMT, CSDNet, Grounding DINO, GLIP, etc. are reported in the [paper](https://arxiv.org/abs/2603.24876) and the [GitHub README](https://github.com/JunDong-dev/OptiSAR-Net-PlusPlus).
55
+
56
+ ## 🚀 Usage
57
+
58
+ `best.pt` contains **custom modules** (`PLoRA_MoE`, `TGDG_SSA`, `OptiSARNetPlusPlusDetect`), so it must be loaded with the project code rather than a stock Ultralytics install:
59
+
60
+ ```bash
61
+ # 1. Clone the official repository (defines the custom modules & inference entry points)
62
+ git clone https://github.com/JunDong-dev/OptiSAR-Net-PlusPlus.git
63
+ cd OptiSAR-Net-PlusPlus
64
+ pip install -r requirements.txt
65
+
66
+ # 2. Download the checkpoint and the dataset
67
+ hf download JunDong-dev/OptiSAR-Net-PlusPlus best.pt --local-dir .
68
+ hf download JunDong-dev/OptSAR-RSVG --repo-type dataset --local-dir OptSAR-RSVG
69
+
70
+ # 3. Run evaluation / inference with the scripts provided in the repository README
71
+ ```
72
+
73
+ Evaluation metrics follow the standard CD-RSVG protocol: Pr@{0.5–0.9}, meanIoU, cumIoU, with per-domain (`optical` / `sar`) reporting.
74
+
75
+ ## 🎯 Intended Use
76
+
77
+ - Cross-domain (optical ↔ SAR) referring expression comprehension / visual grounding in remote sensing
78
+ - Research on multi-modal fusion, parameter-efficient MoE adaptation, and contrastive region–text matching
79
+
80
+ ## ⚠️ Limitations
81
+
82
+ - Trained and evaluated only on the 16 categories of OptSAR-RSVG; out-of-distribution categories/imagery may degrade
83
+ - SAR performance at strict IoU thresholds (Pr@0.9) is notably lower than optical, reflecting SAR imaging characteristics
84
+
85
+ ## ⚖️ License
86
+
87
+ Code and weights are released under **AGPL-3.0**. The companion dataset is subject to its source datasets' licenses (see the dataset card).
88
+
89
+ ## 📚 Citation
90
+
91
+ ```bibtex
92
+ @article{tang2026optisar,
93
+ title={OptiSAR-Net++: A Large-Scale Benchmark and Transformer-Free Framework for Cross-Domain Remote Sensing Visual Grounding},
94
+ author={Tang, Xiaoyu and Dong, Jun and Cheng, Jintao and Fan, Rui},
95
+ journal={arXiv preprint arXiv:2603.24876},
96
+ year={2026}
97
+ }
98
+ ```