card: FGIRFT full fine-tuning benchmark
Browse files
README.md
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: image-classification
|
| 3 |
+
library_name: fgir-zoo
|
| 4 |
+
tags:
|
| 5 |
+
- fine-grained
|
| 6 |
+
- image-classification
|
| 7 |
+
- vision-transformer
|
| 8 |
+
- full-fine-tuning
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# FGIRFT
|
| 12 |
+
|
| 13 |
+
A full fine-tuning benchmark for fine-grained image recognition (FGIR). Every checkpoint is an
|
| 14 |
+
ImageNet-21k **ViT-B/16 fine-tuned end to end** — the whole backbone is trained, not frozen — with a
|
| 15 |
+
fine-grained classification head, across 21 fine-grained datasets. FGIRFT is the full fine-tuning
|
| 16 |
+
counterpart to the ERIS Lab's parameter-efficient works (the ILA/SAW and AAA adapters in
|
| 17 |
+
`ERISLab/FGIR-ViT`, which keep the backbone frozen and train only small adapters).
|
| 18 |
+
|
| 19 |
+
242 checkpoints, one seed per configuration, last epoch only, grouped by input resolution: `ft_224`
|
| 20 |
+
(111) and `ft_448` (131). Everything is described in `manifest.csv` and loadable by name with the
|
| 21 |
+
standalone `fgir_zoo` library (no training repo needed).
|
| 22 |
+
|
| 23 |
+
## What it holds
|
| 24 |
+
|
| 25 |
+
ViT-B/16 fully fine-tuned at 224 or 448 px, each with one FGIR classification head: a plain linear
|
| 26 |
+
classifier (`cls`), GLSim, MAWS (the FFVT head), PSM (the TransFG head), CAL, MPN-COV, and
|
| 27 |
+
attention-rollout heads. The 21 datasets span general fine-grained sets (CUB, NABirds, Aircraft,
|
| 28 |
+
Stanford Cars, Stanford Dogs, Oxford Flowers, Oxford Pets, Food-101, VegFru, DAFB, Moe) and the
|
| 29 |
+
ultra-fine-grained leaf sets (Cotton, SoyAgeing and its five single-season subsets, SoyGene,
|
| 30 |
+
SoyGlobal, SoyLocal). `num_classes` ranges from 37 to 3263. `manifest.csv` maps every file to its
|
| 31 |
+
dataset, head, serial, resolution, `num_classes`, top-1 accuracy and sha256.
|
| 32 |
+
|
| 33 |
+
## Load a checkpoint
|
| 34 |
+
|
| 35 |
+
Standalone, no `fgir_vit` install — the model code is vendored inside `fgir_zoo`:
|
| 36 |
+
|
| 37 |
+
```bash
|
| 38 |
+
pip install git+https://github.com/arkel23/fgir-zoo.git
|
| 39 |
+
```
|
| 40 |
+
|
| 41 |
+
```python
|
| 42 |
+
import fgir_zoo
|
| 43 |
+
model = fgir_zoo.load('fgirvit/ft_448/aircraft_vit_b16_avg_cls_rollout_3')
|
| 44 |
+
model.eval()
|
| 45 |
+
# fgir_zoo.list_models(family='fgirvit', group='ft_448') lists every name; model.config holds the run's settings.
|
| 46 |
+
```
|
| 47 |
+
|
| 48 |
+
Or fetch the file directly and load it yourself:
|
| 49 |
+
|
| 50 |
+
```python
|
| 51 |
+
import torch
|
| 52 |
+
from huggingface_hub import hf_hub_download
|
| 53 |
+
path = hf_hub_download('ERISLab/FGIRFT', 'ft_448/aircraft_vit_b16_avg_cls_rollout_3.pth')
|
| 54 |
+
ckpt = torch.load(path, map_location='cpu', weights_only=False) # dict: config, model, accuracy, epoch
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
## Requirements
|
| 58 |
+
|
| 59 |
+
- `torch>=2.5`, `timm==0.9.12` (the checkpoints were trained on it and the vendored model code forks
|
| 60 |
+
that version), `huggingface_hub`, `safetensors`, `einops`, `ml_collections`.
|
| 61 |
+
|
| 62 |
+
## Related
|
| 63 |
+
|
| 64 |
+
Part of the [ERISLab FGIRFT collection](https://huggingface.co/collections/ERISLab/fgirft-full-fine-tuning-benchmark-for-fgir-6ab2e8940f3a28ec7ba0bfbf).
|
| 65 |
+
For the parameter-efficient adapter works (ILA, SAW, AAA) on a frozen ViT-B/16, see
|
| 66 |
+
[`ERISLab/FGIR-ViT`](https://huggingface.co/ERISLab/FGIR-ViT). As a full fine-tuning line, FGIRFT sits
|
| 67 |
+
close to the [`ERISLab/FGIR-Backbones`](https://huggingface.co/ERISLab/FGIR-Backbones) benchmark,
|
| 68 |
+
which studies backbones and pretraining recipes for fine-grained recognition.
|