language: en
license: other
library_name: pytorch
tags:
- remote-sensing
- satellite-imagery
- earth-observation
- change-detection
- visual-grounding
- image-captioning
- visual-question-answering
- optical-sar-fusion
- lora
- peft
pipeline_tag: image-to-text
config_hash: 78f1e3700da15aa1
Model Card — SatQuery AI
SatQuery AI answers natural-language questions about satellite imagery using a router + specialists design. This card covers the six trained artifacts released by the project. It is deliberately explicit about what is measured, what is not, and what was rejected.
The six trained artifacts are small modules on top of frozen, publicly-pinned backbones. No backbone weights are redistributed by this release — they are fetched from the Hugging Face Hub at run time, pinned by revision.
Machine-readable identities (byte counts and sha256) are in
models/manifest.json and models/checksums.sha256,
generated by reading the files. Where this card and the generated manifest disagree, the manifest
wins.
1. Artifacts in this release
| # | Task | Kind | File | Bytes | sha256 (first 16) |
|---|---|---|---|---|---|
| 1 | change |
trained head | head.pt |
63,231,009 | c5ef31277b67aa01 |
| 2 | change_vqa |
trained head | head.pt |
5,822,809 | cfae5e43b97ca930 |
| 3 | optical_sar |
trained head | head.pt |
14,427,457 | 785815729a3a39fc |
| 4 | grounding |
trained head | head.pt |
12,639,041 | 93432f7034be91a8 |
| 5 | router |
trained adapter | adapter.pt |
211,961 | 8527c3ed28a293e1 |
| 6 | vlm |
LoRA adapter | adapter_model.safetensors |
34,798,048 | 07c76a75fa046248 |
Two of these hashes (change_vqa, vlm) agree exactly with hashes recorded independently at
promotion time — an external cross-check, not a self-consistency claim.
2. Backbone dependencies (frozen, pinned by revision)
| Role | Repository | Revision |
|---|---|---|
| Router encoder | sentence-transformers/all-MiniLM-L6-v2 |
1110a243fdf4 |
| VLM | HuggingFaceTB/SmolVLM-500M-Instruct |
a7da5b986cb5 |
| Grounding | chendelong/RemoteCLIP (RemoteCLIP-ViT-B-32.pt) |
bf1d8a3ccf2d |
| Optical-SAR | antofuller/CROMA (CROMA_base.pt) |
0dd28e3d633b |
| Change | STANet-style (ResNet-18 + PAM) | trained in-project |
3. Intended use
- Research and demonstration of a modular, CPU-first remote-sensing QA system.
- Routing and dispatch of natural-language queries to the appropriate specialist.
- Reproducible evaluation of each specialist on its own documented split.
4. Out-of-scope use
- Safety-, legal- or life-critical decisions. No accuracy, calibration or robustness guarantee is offered for any high-stakes use.
- Operational geospatial production without independent validation.
- Any use of the VLM adapter as a production model — it is acceptance-rejected (§6).
- Treating per-specialist metrics as system-level accuracy. No end-to-end benchmark exists (§7).
5. Measured performance
| Task | Metric | Value | Split / protocol |
|---|---|---|---|
| change | pooled IoU / macro IoU / pooled F1 | 0.8122 / 0.8457 / 0.8964 | LEVIR-CD-256 test, n = 2048 |
| grounding | mean best IoU / recall@0.5 | 0.2838 / 0.2198 canonical; 0.2566 / 0.1938 matched6 | VRSBench, n = 16159 |
| grounding | head-argmax / zero-shot baseline IoU | 0.1215 / 0.0972 | canonical |
| optical_sar | accuracy / macro F1 | 0.931 / 0.434161 | held-out test, n = 4000, 19 classes |
| change_vqa | accuracy / macro F1 | 0.697626 / 0.378373 (test); 0.651469 / 0.372309 (test2) | two test sets |
| vlm | exact_match / F1 | 0.963 / 0.96432 | frozen 1000-question subset |
| router | overall ungated accuracy | 0.965116 | val, n = 86 — TEST NOT RUN |
Every value is checked against its artifact by tools/verify_readme_metrics.py.
5.1 Calibration — reported as a negative result
Temperature scaling is enabled (T = 0.9772732, fit on val n = 16,441). ECE worsened: 0.013755 → 0.014929. It is retained because it is part of the frozen configuration, not because it helped.
6. Acceptance status
| Artifact | Metrics | Acceptance |
|---|---|---|
| change | VERIFIED | accepted (shipped) |
| grounding | measured (2 protocols) | shipped |
| optical_sar | measured | ruling OPEN |
| change_vqa | measured (2 test sets) | ruling OPEN |
| router | measured (val only) | shipped; test NOT RUN |
| vlm | usable (exact_match 0.963) | ACCEPTANCE-REJECTED |
USABLE_VERIFIED ≠ACCEPTANCE-ACCEPTED. The VLM adapter works and is not promoted; the deployed
caption/VQA path uses the unadapted model.
7. Evaluation gaps (stated, not hidden)
- No system-level end-to-end benchmark exists. None is claimed.
- Router test split: NOT RUN.
- Benchmark adapters: NOT RUN.
- Cross-dataset generalisation: NOT RUN.
- Human and robustness evaluation: NOT RUN.
8. Limitations
- Grounding absolute IoU is low (0.28) and protocol-sensitive.
- Optical-SAR accuracy is carried by common classes (macro-F1 0.434161).
- The BigEarthNet local subset is 100 % single-label vs the official 1–11 multi-label scheme, so its metrics are not comparable to published numbers.
- The optical-SAR service returns a bare class index, not a label.
- Known router residuals exist (e.g. "What is the new runway?" reads
change). - No
LICENSEfile exists in the source repository.
See docs/LIMITATIONS.md for the full catalogue.
9. Training summary
Small modules on frozen backbones; seed 42; every artifact records the frozen config hash
78f1e3700da15aa1. Router and change/grounding/fusion heads train on CPU; the VLM LoRA adapter and
the change-VQA head were trained on external GPUs (the latter via a documented Kaggle run). Full
detail in docs/TRAINING.md.
10. Provenance and verification
| Item | Location |
|---|---|
| Byte-exact manifest | models/manifest.json |
| Checksums | models/checksums.sha256 |
| Metric verification tool | tools/verify_readme_metrics.py |
| Metric verification output | tools/readme_metrics_report.txt |
| Full documentation | docs/ |
| Release manifest | RELEASE_MANIFEST.md |
11. Licence
The project ships no licence file; a licence must be selected by the owner before public release of the code. Model weights carry the terms of their backbone licences — consult each backbone's Hugging Face page. Backbones are not redistributed here.
12. Citation
If you use this work, cite the project repository:
@misc{satquery_ai_2026,
title = {SatQuery AI: A Modular Router-and-Specialists System for Satellite Imagery Question Answering},
author = {SatQuery AI},
year = {2026},
note = {Public release: https://github.com/Anish-lab-blip/SatQuery-AI}
}