linxin02 commited on
Commit
3bc71d8
Β·
verified Β·
1 Parent(s): 4699b1a

Upload metrics/METRICS_25K_SNAPSHOT.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. metrics/METRICS_25K_SNAPSHOT.md +16 -12
metrics/METRICS_25K_SNAPSHOT.md CHANGED
@@ -1,8 +1,8 @@
1
  # DeCo GAN-Finetune β€” 25k-step Snapshot
2
 
3
  All finetunes compared at the **same 25 000-step checkpoint**; `original` (pretrained DeCo, no finetune)
4
- is the reference. Methods that have **not yet reached 25k are excluded** for now: `gan_dinov2_t035`
5
- (only to 20k) and `drift` (only to 15k) β€” add once they hit 25k.
6
 
7
  Arrows: ↑ higher-better, ↓ lower-better. `β€”` = not measured for that model. `n=10000` for COCO & quality.
8
 
@@ -12,25 +12,28 @@ Arrows: ↑ higher-better, ↓ lower-better. `β€”` = not measured for that model
12
  | baseline | 0.8160 | 0.8564 | 36.818 | 36.982 | 0.8097 | 0.4156 | 0.7062 | 75.464 | 0.6320 | 29.162 |
13
  | gan_patchgan | 0.8166 | 0.8581 | 37.103 | 37.029 | 0.8149 | 0.3961 | 0.7017 | 75.422 | 0.6270 | 29.288 |
14
  | gan_patchgan_bn | 0.8169 | 0.8570 | β€” | β€” | β€” | β€” | β€” | β€” | β€” | β€” |
15
- | gan_dinov2 | 0.8285 | 0.8567 | 34.887 | **38.989** | 0.7898 | **0.5105** | **0.7803** | 76.373 | 0.7149 | β€” |
16
- | gan_dinov2_50 | 0.8306 | 0.8576 | 34.817 | 36.807 | 0.7805 | 0.4745 | 0.7750 | **76.640** | **0.7217** | 27.319 |
17
  | gan_dinov2_all | 0.8261 | 0.8542 | β€” | β€” | β€” | β€” | β€” | β€” | β€” | β€” |
18
- | **gan_dinov2_text** | **0.8327** | 0.8515 | **32.476** | 37.202 | **0.7246** | 0.4784 | 0.7666 | 76.489 | 0.7100 | **25.526** |
 
19
  | gan_dinov2_50_longcap | 0.8144 | 0.7113 | β€” | β€” | β€” | β€” | β€” | β€” | β€” | β€” |
20
 
21
  **Notes**
22
  - `original` has no "25k" (it is the base model); its own values are used as the reference row. Its DPG
23
  (~0.8158) was measured on the pretrained checkpoint (not stored per-step on HF).
24
- - COCO-10K measured for: original, baseline, gan_patchgan, gan_dinov2, gan_dinov2_50, gan_dinov2_text (@25k) β€” others `β€”`.
25
  - Visual quality measured for: original, baseline, gan_patchgan, gan_dinov2 (NR-IQA only), gan_dinov2_50,
26
- gan_dinov2_text β€” `gan_dinov2` pFID still missing.
27
 
28
  **Read at a glance (25k)**
29
- - **DPG**: DINOv2-disc GANs win; `gan_dinov2_text` best (0.8327), then dinov2_50 (0.8306) > dinov2 (0.8285)
30
- > dinov2_all (0.8261). baseline/patchgan/patchgan_bn ~0.816; longcap collapsing (0.8144).
31
  - **GenEval**: all ~0.85–0.86 (GAN β‰ˆ baseline; realism, not composition); longcap crashed to 0.7113.
32
- - **COCO realism**: DINOv2 GANs best FID/IS/Recall; patchgan worst FID (37.1, above original).
33
- - **Visual quality**: original β‰ͺ baseline β‰ˆ patchgan β‰ͺ DINOv2 GANs; best pFID = gan_dinov2_text (25.53).
 
 
34
 
35
  ## P-IDS / U-IDS @25k (CoModGAN realism; higher-better, max 0.5; side-signal only β€” very noisy near floor)
36
 
@@ -41,8 +44,9 @@ Arrows: ↑ higher-better, ↓ lower-better. `β€”` = not measured for that model
41
  | gan_patchgan | 0.0077 | 0.0002 |
42
  | **gan_dinov2** | **0.0235** | **0.0030** |
43
  | gan_dinov2_text | 0.0152 | 0.0012 |
 
44
  | gan_dinov2_50 | 0.0038 | 0.0000 |
45
 
46
  gan_dinov2 most realistic here (β‰ˆ3Γ— others), then gan_dinov2_text; absolute values tiny (SVM still
47
  separates real vs gen ~98%+). Precision (dropped from main table) @25k: patchgan 0.407 > baseline 0.402 >
48
- original 0.380 > gan_dinov2 0.351 > dinov2_50 0.323 > drift(15k) 0.301 β€” inverse of realism (P/R tradeoff).
 
1
  # DeCo GAN-Finetune β€” 25k-step Snapshot
2
 
3
  All finetunes compared at the **same 25 000-step checkpoint**; `original` (pretrained DeCo, no finetune)
4
+ is the reference. `gan_dinov2_t035` now reaches 25k and is included (full suite). Only `drift` remains
5
+ excluded (its single ckpt is at 15k).
6
 
7
  Arrows: ↑ higher-better, ↓ lower-better. `β€”` = not measured for that model. `n=10000` for COCO & quality.
8
 
 
12
  | baseline | 0.8160 | 0.8564 | 36.818 | 36.982 | 0.8097 | 0.4156 | 0.7062 | 75.464 | 0.6320 | 29.162 |
13
  | gan_patchgan | 0.8166 | 0.8581 | 37.103 | 37.029 | 0.8149 | 0.3961 | 0.7017 | 75.422 | 0.6270 | 29.288 |
14
  | gan_patchgan_bn | 0.8169 | 0.8570 | β€” | β€” | β€” | β€” | β€” | β€” | β€” | β€” |
15
+ | gan_dinov2 | 0.8285 | 0.8567 | 34.887 | **38.989** | 0.7898 | **0.5105** | 0.7803 | 76.373 | 0.7149 | β€” |
16
+ | gan_dinov2_50 | 0.8306 | 0.8576 | 34.817 | 36.807 | 0.7805 | 0.4745 | 0.7750 | **76.640** | 0.7217 | 27.319 |
17
  | gan_dinov2_all | 0.8261 | 0.8542 | β€” | β€” | β€” | β€” | β€” | β€” | β€” | β€” |
18
+ | **gan_dinov2_t035** | **0.8337** | 0.8575 | 34.324 | 37.978 | 0.8092 | 0.4976 | **0.7827** | 76.577 | **0.7263** | 27.661 |
19
+ | gan_dinov2_text | 0.8327 | 0.8515 | **32.476** | 37.202 | **0.7246** | 0.4784 | 0.7666 | 76.489 | 0.7100 | **25.526** |
20
  | gan_dinov2_50_longcap | 0.8144 | 0.7113 | β€” | β€” | β€” | β€” | β€” | β€” | β€” | β€” |
21
 
22
  **Notes**
23
  - `original` has no "25k" (it is the base model); its own values are used as the reference row. Its DPG
24
  (~0.8158) was measured on the pretrained checkpoint (not stored per-step on HF).
25
+ - COCO-10K measured for: original, baseline, gan_patchgan, gan_dinov2, gan_dinov2_50, gan_dinov2_t035, gan_dinov2_text (@25k) β€” others `β€”`.
26
  - Visual quality measured for: original, baseline, gan_patchgan, gan_dinov2 (NR-IQA only), gan_dinov2_50,
27
+ gan_dinov2_t035, gan_dinov2_text β€” `gan_dinov2` pFID still missing.
28
 
29
  **Read at a glance (25k)**
30
+ - **DPG**: DINOv2-disc GANs win; `gan_dinov2_t035` best (0.8337), then gan_dinov2_text (0.8327) > dinov2_50
31
+ (0.8306) > dinov2 (0.8285) > dinov2_all (0.8261). baseline/patchgan/patchgan_bn ~0.816; longcap 0.8144.
32
  - **GenEval**: all ~0.85–0.86 (GAN β‰ˆ baseline; realism, not composition); longcap crashed to 0.7113.
33
+ - **COCO realism**: `gan_dinov2_text` best FID (32.5) + CMMD (0.725); `gan_dinov2` best IS/Recall; patchgan
34
+ worst FID. Note `t035` has strong FID (34.3) but **worst CMMD (0.809, β‰ˆbaseline)** despite top DPG.
35
+ - **Visual quality**: original β‰ͺ baseline β‰ˆ patchgan β‰ͺ DINOv2 GANs. `t035` best NR-IQA (TOPIQ 0.783 / MANIQA
36
+ 0.726); `gan_dinov2_text` best pFID (25.53).
37
 
38
  ## P-IDS / U-IDS @25k (CoModGAN realism; higher-better, max 0.5; side-signal only β€” very noisy near floor)
39
 
 
44
  | gan_patchgan | 0.0077 | 0.0002 |
45
  | **gan_dinov2** | **0.0235** | **0.0030** |
46
  | gan_dinov2_text | 0.0152 | 0.0012 |
47
+ | gan_dinov2_t035 | 0.0083 | 0.0001 |
48
  | gan_dinov2_50 | 0.0038 | 0.0000 |
49
 
50
  gan_dinov2 most realistic here (β‰ˆ3Γ— others), then gan_dinov2_text; absolute values tiny (SVM still
51
  separates real vs gen ~98%+). Precision (dropped from main table) @25k: patchgan 0.407 > baseline 0.402 >
52
+ original 0.380 > gan_dinov2 0.351 > t035 0.334 > dinov2_50 0.323 > drift(15k) 0.301 β€” inverse of realism (P/R tradeoff).