oonat commited on
Commit
e99eb83
Β·
verified Β·
1 Parent(s): 1592e64

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +122 -0
README.md CHANGED
@@ -1,3 +1,125 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ library_name: pytorch
4
+ tags:
5
+ - clip
6
+ - siglip
7
+ - zero-shot-image-classification
8
+ - interpretability
9
+ - concept-bottleneck
10
+ - vision-language
11
+ - cvpr2026
12
+ datasets:
13
+ - oonat/ezpc-embeddings
14
+ base_model:
15
+ - openai/clip-rn50
16
+ - openai/clip-vit-base-patch32
17
+ - openai/clip-vit-large-patch14
18
+ - google/siglip-so400m-patch14-384
19
+ pipeline_tag: zero-shot-image-classification
20
  ---
21
+
22
+ # EZPC - Pre-trained Concept Projection Matrices
23
+
24
+ This repository hosts the trained projection matrices **A** for **Explaining CLIP Zero-shot Predictions Through Concepts** (CVPR 2026).
25
+
26
+ - πŸ“„ **Paper:** [arXiv:2603.28211](https://arxiv.org/abs/2603.28211)
27
+ - πŸ’» **Code:** [github.com/oonat/ezpc](https://github.com/oonat/ezpc)
28
+ - 🌐 **Project page:** [oonat.github.io/ezpc](https://oonat.github.io/ezpc)
29
+ - πŸ€— **Embeddings:** [oonat/ezpc-embeddings](https://huggingface.co/datasets/oonat/ezpc-embeddings)
30
+
31
+ ## Repository Layout
32
+
33
+ Each checkpoint is a single PyTorch tensor file (`best_A.pth`) inside a folder whose name encodes the training configuration:
34
+
35
+ ```
36
+ checkpoints/
37
+ └── <dataset>_backbone_<backbone>_weight_<Ξ»>_epoch_<E>_lr_<lr>_bs_<bs>/
38
+ └── best_A.pth
39
+ ```
40
+
41
+ For example:
42
+
43
+ ```
44
+ checkpoints/CIFAR-100_backbone_RN50_weight_1.0_epoch_10000_lr_0.01_bs_1000000/best_A.pth
45
+ ```
46
+
47
+ The tensor `best_A.pth` has shape **(d, m)** where *d* is the backbone embedding dimension and *m* is the number of concepts in the dataset's concept vocabulary.
48
+
49
+ ## Quickstart
50
+
51
+ ### 1. Download a checkpoint
52
+
53
+ ```bash
54
+ pip install huggingface-hub
55
+
56
+ # Download all checkpoints
57
+ hf download oonat/ezpc-checkpoints \
58
+ --local-dir . \
59
+ --include "checkpoints/*"
60
+
61
+ # Or just one
62
+ hf download oonat/ezpc-checkpoints \
63
+ --local-dir . \
64
+ --include "checkpoints/CIFAR-100_backbone_RN50_weight_1.0_epoch_10000_lr_0.01_bs_1000000/*"
65
+ ```
66
+
67
+ ### 2. Load and use it
68
+
69
+ The checkpoint can be loaded directly with PyTorch:
70
+
71
+ ```python
72
+ import torch
73
+
74
+ A = torch.load(
75
+ "checkpoints/CIFAR-100_backbone_RN50_weight_1.0_epoch_10000_lr_0.01_bs_1000000/best_A.pth",
76
+ weights_only=True,
77
+ ).float()
78
+ print(A.shape) # (d, m)
79
+ ```
80
+
81
+ To run zero-shot evaluation, qualitative concept visualizations, faithfulness
82
+ analyses, or any of the experiments from the paper, clone the
83
+ [EZPC GitHub repo](https://github.com/oonat/ezpc).
84
+
85
+ Evaluation also requires the pre-computed image **and** cached text embeddings.
86
+ Download them from the [embeddings dataset](https://huggingface.co/datasets/oonat/ezpc-embeddings)
87
+ into `./data`:
88
+
89
+ ```bash
90
+ hf download oonat/ezpc-embeddings --repo-type dataset --local-dir data
91
+ ```
92
+
93
+ Then point `--checkpoint_path` at the downloaded `best_A.pth`:
94
+
95
+ ```bash
96
+ python test.py \
97
+ --dataset CIFAR-100 \
98
+ --dataset_root ./data \
99
+ --backbone RN50 \
100
+ --checkpoint_path ./checkpoints/CIFAR-100_backbone_RN50_weight_1.0_epoch_10000_lr_0.01_bs_1000000/best_A.pth
101
+ ```
102
+
103
+ ## Citation
104
+
105
+ ```bibtex
106
+ @InProceedings{Ozdemir_2026_CVPR,
107
+ author = {Ozdemir, Onat and Christensen, Anders and Alaniz, Stephan and Akata, Zeynep and Akbas, Emre},
108
+ title = {Explaining CLIP Zero-shot Predictions Through Concepts},
109
+ booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
110
+ month = {June},
111
+ year = {2026},
112
+ pages = {31336-31345}
113
+ }
114
+ ```
115
+
116
+ ## Acknowledgements
117
+
118
+ The concept vocabularies and class label mapping files were originally curated by the [Label-free Concept Bottleneck Models](https://github.com/Trustworthy-ML-Lab/Label-free-CBM) authors. We thank them for open-sourcing these resources.
119
+
120
+ ## License
121
+
122
+ Released under the MIT License.
123
+
124
+ Note that these checkpoints were trained on embeddings derived from CIFAR-100, CUB-200-2011, Places365, ImageNet, and ImageNet-100. Users are responsible for complying with the original license and terms of use of those datasets, which may restrict commercial use β€” notably ImageNet and CUB-200-2011, which are released for non-commercial research only.
125
+