Initial upload
Browse files- README.md +76 -0
- distribution_shift/imagenet_b16/config.json +3 -0
- distribution_shift/imagenet_b16/model.safetensors +3 -0
- distribution_shift/imagenet_b32/config.json +3 -0
- distribution_shift/imagenet_b32/model.safetensors +3 -0
- distribution_shift/imagenet_petl_b16/config.json +3 -0
- distribution_shift/imagenet_petl_b16/model.safetensors +3 -0
- distribution_shift/imagenet_petl_b32/config.json +3 -0
- distribution_shift/imagenet_petl_b32/model.safetensors +3 -0
- distribution_shift/iwildcam_b16/config.json +3 -0
- distribution_shift/iwildcam_b16/model.safetensors +3 -0
- distribution_shift/iwildcam_b32/config.json +3 -0
- distribution_shift/iwildcam_b32/model.safetensors +3 -0
- distribution_shift/iwildcam_petl_b16/config.json +3 -0
- distribution_shift/iwildcam_petl_b16/model.safetensors +3 -0
- distribution_shift/iwildcam_petl_b32/config.json +3 -0
- distribution_shift/iwildcam_petl_b32/model.safetensors +3 -0
- transfer/caltech101/config.json +3 -0
- transfer/caltech101/model.safetensors +3 -0
- transfer/flowers102/config.json +3 -0
- transfer/flowers102/model.safetensors +3 -0
- transfer/pcam/config.json +3 -0
- transfer/pcam/model.safetensors +3 -0
- transfer/stanfordcars/config.json +3 -0
- transfer/stanfordcars/model.safetensors +3 -0
README.md
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model:
|
| 4 |
+
- openai/clip-vit-base-patch16
|
| 5 |
+
- openai/clip-vit-base-patch32
|
| 6 |
+
pipeline_tag: zero-shot-image-classification
|
| 7 |
+
tags:
|
| 8 |
+
- clip
|
| 9 |
+
- vision-language
|
| 10 |
+
- prompt-learning
|
| 11 |
+
- contrastive-learning
|
| 12 |
+
- distribution-shift
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# ReCalCon
|
| 16 |
+
|
| 17 |
+
Checkpoints for **Re-calibrated Contrastive Loss for Transformation-Aware Prompt Conditioning in Vision-Language
|
| 18 |
+
Models** (BMVC 2026). [Paper](https://arxiv.org/abs/2609.06967) · [Code](https://github.com/SoongE/ReCalCon)
|
| 19 |
+
|
| 20 |
+
Each folder holds `model.safetensors` (the full model state dict, fp32) and `config.json`, which names the released
|
| 21 |
+
config to evaluate it with.
|
| 22 |
+
|
| 23 |
+
## Usage
|
| 24 |
+
|
| 25 |
+
```bash
|
| 26 |
+
hf download SoongE/ReCalCon --include "distribution_shift/imagenet_b16/*" --local-dir checkpoints
|
| 27 |
+
python -m scripts.eval --config imagenet_b16 \
|
| 28 |
+
--eval.checkpoint checkpoints/distribution_shift/imagenet_b16/model.safetensors
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
## Checkpoints
|
| 32 |
+
|
| 33 |
+
**Distribution shift** (top-1 accuracy, %)
|
| 34 |
+
|
| 35 |
+
| Checkpoint | Row | ImageNet | -R | -A | -V2 | -Sketch | ObjectNet |
|
| 36 |
+
| --- | --- | --- | --- | --- | --- | --- | --- |
|
| 37 |
+
| `distribution_shift/imagenet_b16` | ViT-B/16, Update, ImageNet | 83.284 | 71.607 | 54.893 | 74.1 | 51.781 | 58.388 |
|
| 38 |
+
| `distribution_shift/imagenet_b32` | ViT-B/32, Update, ImageNet | 79.354 | 62.41 | 32.573 | 68.15 | 44.503 | 49.203 |
|
| 39 |
+
| `distribution_shift/imagenet_petl_b16` | ViT-B/16, Freeze (PETL), ImageNet | 80.206 | 73.163 | 50.867 | 70.13 | 49.614 | 55.691 |
|
| 40 |
+
| `distribution_shift/imagenet_petl_b32` | ViT-B/32, Freeze (PETL), ImageNet | 76.12 | 63.563 | 30.453 | 64.52 | 42.267 | 46.662 |
|
| 41 |
+
|
| 42 |
+
iWildCam (macro-F1, %). The released evaluation reports the OOD split.
|
| 43 |
+
|
| 44 |
+
| Checkpoint | Row | ID (macro-F1) | OOD (macro-F1) |
|
| 45 |
+
| --- | --- | --- | --- |
|
| 46 |
+
| `distribution_shift/iwildcam_b16` | ViT-B/16, Update, iWildCam | 51.253 | 37.803 |
|
| 47 |
+
| `distribution_shift/iwildcam_b32` | ViT-B/32, Update, iWildCam | 42.116 | 29.275 |
|
| 48 |
+
| `distribution_shift/iwildcam_petl_b16` | ViT-B/16, Freeze (PETL), iWildCam | 45.49 | 30.327 |
|
| 49 |
+
| `distribution_shift/iwildcam_petl_b32` | ViT-B/32, Freeze (PETL), iWildCam | 34.709 | 24.502 |
|
| 50 |
+
|
| 51 |
+
**Transfer learning** (top-1 accuracy, %)
|
| 52 |
+
|
| 53 |
+
| Checkpoint | Dataset | Accuracy |
|
| 54 |
+
| --- | --- | --- |
|
| 55 |
+
| `transfer/caltech101` | Caltech101 | 97.85 |
|
| 56 |
+
| `transfer/flowers102` | Flowers102 | 98.894 |
|
| 57 |
+
| `transfer/pcam` | PCam | 89.413 |
|
| 58 |
+
| `transfer/stanfordcars` | StanfordCars | 91.419 |
|
| 59 |
+
|
| 60 |
+
## License
|
| 61 |
+
|
| 62 |
+
Apache License 2.0. The models are fine-tuned from OpenAI CLIP (MIT License); each evaluation dataset keeps its own
|
| 63 |
+
terms.
|
| 64 |
+
|
| 65 |
+
## Citation
|
| 66 |
+
|
| 67 |
+
```bibtex
|
| 68 |
+
@inproceedings{oh2026recalibrated,
|
| 69 |
+
title = {Re-calibrated Contrastive Loss for Transformation-Aware Prompt Conditioning in Vision-Language Models},
|
| 70 |
+
author = {Oh, Seungmin and Kang, Seunghun and Ryu, Jongbin},
|
| 71 |
+
booktitle = {Proceedings of the British Machine Vision Conference},
|
| 72 |
+
year = {2026},
|
| 73 |
+
publisher = {BMVA},
|
| 74 |
+
url = {https://arxiv.org/abs/2609.06967}
|
| 75 |
+
}
|
| 76 |
+
```
|
distribution_shift/imagenet_b16/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "imagenet_b16"
|
| 3 |
+
}
|
distribution_shift/imagenet_b16/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:809660e5076c1c3a37d0f0e0f7b8f005789ba1dcb4e93da69d8470ac31917bbd
|
| 3 |
+
size 615332308
|
distribution_shift/imagenet_b32/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "imagenet_b32"
|
| 3 |
+
}
|
distribution_shift/imagenet_b32/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:414e73f903249d20d36275a4c856226698e28827e989eb72fda2d4571b4a66b5
|
| 3 |
+
size 621958612
|
distribution_shift/imagenet_petl_b16/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "imagenet_petl_b16"
|
| 3 |
+
}
|
distribution_shift/imagenet_petl_b16/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:651284eb228862b6147d544c3b8302deceeb03c0ceff6317495af82def5363d8
|
| 3 |
+
size 615332308
|
distribution_shift/imagenet_petl_b32/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "imagenet_petl_b32"
|
| 3 |
+
}
|
distribution_shift/imagenet_petl_b32/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ff7be0aba87cf5c3baf86e4dcbf39699e9d9c037c2bbadb085b2f3fbf4c39e95
|
| 3 |
+
size 621958612
|
distribution_shift/iwildcam_b16/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "iwildcam_b16"
|
| 3 |
+
}
|
distribution_shift/iwildcam_b16/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1ac910d11f5c68e19e33ff721fd39cb48a45556de7033b17446716ccc44d5779
|
| 3 |
+
size 615332308
|
distribution_shift/iwildcam_b32/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "iwildcam_b32"
|
| 3 |
+
}
|
distribution_shift/iwildcam_b32/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0da01920c19eba317317bfa7a1977f50ef26d25b1e98c07d12aab163842a1d87
|
| 3 |
+
size 621958612
|
distribution_shift/iwildcam_petl_b16/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "iwildcam_petl_b16"
|
| 3 |
+
}
|
distribution_shift/iwildcam_petl_b16/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6d38c8b6cb8fda394b5cd33a7befbbb0b0c087c841b1211405a1cac6e68920c4
|
| 3 |
+
size 615332308
|
distribution_shift/iwildcam_petl_b32/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "iwildcam_petl_b32"
|
| 3 |
+
}
|
distribution_shift/iwildcam_petl_b32/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7ff3113bacba5e1b34aa8c6f1bd30278047b15fbda6eca520b72cc75906a6a81
|
| 3 |
+
size 621958612
|
transfer/caltech101/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "transfer_caltech101"
|
| 3 |
+
}
|
transfer/caltech101/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:25398bb69e7b057fbd4ef9136371090725e86a07966f73f87576a16399fcd678
|
| 3 |
+
size 615332308
|
transfer/flowers102/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "transfer_flowers102"
|
| 3 |
+
}
|
transfer/flowers102/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ab67c99558bd5d7df1e160f32a03324599f2ea3ccd0ad6e2941a3349d99364bb
|
| 3 |
+
size 615332308
|
transfer/pcam/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "transfer_pcam"
|
| 3 |
+
}
|
transfer/pcam/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1c49d7896cc8e053e262ab2a90312927dfab796a7913ffb81e692946559c281a
|
| 3 |
+
size 615332308
|
transfer/stanfordcars/config.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"config": "transfer_stanfordcars"
|
| 3 |
+
}
|
transfer/stanfordcars/model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cf0da0bf1f637747bc5a145ba25379b3b3dc6ba3f3e6166961a58a394cca2733
|
| 3 |
+
size 615332308
|