Upload metrics/METRICS_25K_SNAPSHOT.md with huggingface_hub
Browse files- metrics/METRICS_25K_SNAPSHOT.md +16 -12
metrics/METRICS_25K_SNAPSHOT.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
# DeCo GAN-Finetune β 25k-step Snapshot
|
| 2 |
|
| 3 |
All finetunes compared at the **same 25 000-step checkpoint**; `original` (pretrained DeCo, no finetune)
|
| 4 |
-
is the reference.
|
| 5 |
-
(
|
| 6 |
|
| 7 |
Arrows: β higher-better, β lower-better. `β` = not measured for that model. `n=10000` for COCO & quality.
|
| 8 |
|
|
@@ -12,25 +12,28 @@ Arrows: β higher-better, β lower-better. `β` = not measured for that model
|
|
| 12 |
| baseline | 0.8160 | 0.8564 | 36.818 | 36.982 | 0.8097 | 0.4156 | 0.7062 | 75.464 | 0.6320 | 29.162 |
|
| 13 |
| gan_patchgan | 0.8166 | 0.8581 | 37.103 | 37.029 | 0.8149 | 0.3961 | 0.7017 | 75.422 | 0.6270 | 29.288 |
|
| 14 |
| gan_patchgan_bn | 0.8169 | 0.8570 | β | β | β | β | β | β | β | β |
|
| 15 |
-
| gan_dinov2 | 0.8285 | 0.8567 | 34.887 | **38.989** | 0.7898 | **0.5105** |
|
| 16 |
-
| gan_dinov2_50 | 0.8306 | 0.8576 | 34.817 | 36.807 | 0.7805 | 0.4745 | 0.7750 | **76.640** |
|
| 17 |
| gan_dinov2_all | 0.8261 | 0.8542 | β | β | β | β | β | β | β | β |
|
| 18 |
-
| **
|
|
|
|
| 19 |
| gan_dinov2_50_longcap | 0.8144 | 0.7113 | β | β | β | β | β | β | β | β |
|
| 20 |
|
| 21 |
**Notes**
|
| 22 |
- `original` has no "25k" (it is the base model); its own values are used as the reference row. Its DPG
|
| 23 |
(~0.8158) was measured on the pretrained checkpoint (not stored per-step on HF).
|
| 24 |
-
- COCO-10K measured for: original, baseline, gan_patchgan, gan_dinov2, gan_dinov2_50, gan_dinov2_text (@25k) β others `β`.
|
| 25 |
- Visual quality measured for: original, baseline, gan_patchgan, gan_dinov2 (NR-IQA only), gan_dinov2_50,
|
| 26 |
-
gan_dinov2_text β `gan_dinov2` pFID still missing.
|
| 27 |
|
| 28 |
**Read at a glance (25k)**
|
| 29 |
-
- **DPG**: DINOv2-disc GANs win; `
|
| 30 |
-
> dinov2_all (0.8261). baseline/patchgan/patchgan_bn ~0.816; longcap
|
| 31 |
- **GenEval**: all ~0.85β0.86 (GAN β baseline; realism, not composition); longcap crashed to 0.7113.
|
| 32 |
-
- **COCO realism**:
|
| 33 |
-
|
|
|
|
|
|
|
| 34 |
|
| 35 |
## P-IDS / U-IDS @25k (CoModGAN realism; higher-better, max 0.5; side-signal only β very noisy near floor)
|
| 36 |
|
|
@@ -41,8 +44,9 @@ Arrows: β higher-better, β lower-better. `β` = not measured for that model
|
|
| 41 |
| gan_patchgan | 0.0077 | 0.0002 |
|
| 42 |
| **gan_dinov2** | **0.0235** | **0.0030** |
|
| 43 |
| gan_dinov2_text | 0.0152 | 0.0012 |
|
|
|
|
| 44 |
| gan_dinov2_50 | 0.0038 | 0.0000 |
|
| 45 |
|
| 46 |
gan_dinov2 most realistic here (β3Γ others), then gan_dinov2_text; absolute values tiny (SVM still
|
| 47 |
separates real vs gen ~98%+). Precision (dropped from main table) @25k: patchgan 0.407 > baseline 0.402 >
|
| 48 |
-
original 0.380 > gan_dinov2 0.351 > dinov2_50 0.323 > drift(15k) 0.301 β inverse of realism (P/R tradeoff).
|
|
|
|
| 1 |
# DeCo GAN-Finetune β 25k-step Snapshot
|
| 2 |
|
| 3 |
All finetunes compared at the **same 25 000-step checkpoint**; `original` (pretrained DeCo, no finetune)
|
| 4 |
+
is the reference. `gan_dinov2_t035` now reaches 25k and is included (full suite). Only `drift` remains
|
| 5 |
+
excluded (its single ckpt is at 15k).
|
| 6 |
|
| 7 |
Arrows: β higher-better, β lower-better. `β` = not measured for that model. `n=10000` for COCO & quality.
|
| 8 |
|
|
|
|
| 12 |
| baseline | 0.8160 | 0.8564 | 36.818 | 36.982 | 0.8097 | 0.4156 | 0.7062 | 75.464 | 0.6320 | 29.162 |
|
| 13 |
| gan_patchgan | 0.8166 | 0.8581 | 37.103 | 37.029 | 0.8149 | 0.3961 | 0.7017 | 75.422 | 0.6270 | 29.288 |
|
| 14 |
| gan_patchgan_bn | 0.8169 | 0.8570 | β | β | β | β | β | β | β | β |
|
| 15 |
+
| gan_dinov2 | 0.8285 | 0.8567 | 34.887 | **38.989** | 0.7898 | **0.5105** | 0.7803 | 76.373 | 0.7149 | β |
|
| 16 |
+
| gan_dinov2_50 | 0.8306 | 0.8576 | 34.817 | 36.807 | 0.7805 | 0.4745 | 0.7750 | **76.640** | 0.7217 | 27.319 |
|
| 17 |
| gan_dinov2_all | 0.8261 | 0.8542 | β | β | β | β | β | β | β | β |
|
| 18 |
+
| **gan_dinov2_t035** | **0.8337** | 0.8575 | 34.324 | 37.978 | 0.8092 | 0.4976 | **0.7827** | 76.577 | **0.7263** | 27.661 |
|
| 19 |
+
| gan_dinov2_text | 0.8327 | 0.8515 | **32.476** | 37.202 | **0.7246** | 0.4784 | 0.7666 | 76.489 | 0.7100 | **25.526** |
|
| 20 |
| gan_dinov2_50_longcap | 0.8144 | 0.7113 | β | β | β | β | β | β | β | β |
|
| 21 |
|
| 22 |
**Notes**
|
| 23 |
- `original` has no "25k" (it is the base model); its own values are used as the reference row. Its DPG
|
| 24 |
(~0.8158) was measured on the pretrained checkpoint (not stored per-step on HF).
|
| 25 |
+
- COCO-10K measured for: original, baseline, gan_patchgan, gan_dinov2, gan_dinov2_50, gan_dinov2_t035, gan_dinov2_text (@25k) β others `β`.
|
| 26 |
- Visual quality measured for: original, baseline, gan_patchgan, gan_dinov2 (NR-IQA only), gan_dinov2_50,
|
| 27 |
+
gan_dinov2_t035, gan_dinov2_text β `gan_dinov2` pFID still missing.
|
| 28 |
|
| 29 |
**Read at a glance (25k)**
|
| 30 |
+
- **DPG**: DINOv2-disc GANs win; `gan_dinov2_t035` best (0.8337), then gan_dinov2_text (0.8327) > dinov2_50
|
| 31 |
+
(0.8306) > dinov2 (0.8285) > dinov2_all (0.8261). baseline/patchgan/patchgan_bn ~0.816; longcap 0.8144.
|
| 32 |
- **GenEval**: all ~0.85β0.86 (GAN β baseline; realism, not composition); longcap crashed to 0.7113.
|
| 33 |
+
- **COCO realism**: `gan_dinov2_text` best FID (32.5) + CMMD (0.725); `gan_dinov2` best IS/Recall; patchgan
|
| 34 |
+
worst FID. Note `t035` has strong FID (34.3) but **worst CMMD (0.809, βbaseline)** despite top DPG.
|
| 35 |
+
- **Visual quality**: original βͺ baseline β patchgan βͺ DINOv2 GANs. `t035` best NR-IQA (TOPIQ 0.783 / MANIQA
|
| 36 |
+
0.726); `gan_dinov2_text` best pFID (25.53).
|
| 37 |
|
| 38 |
## P-IDS / U-IDS @25k (CoModGAN realism; higher-better, max 0.5; side-signal only β very noisy near floor)
|
| 39 |
|
|
|
|
| 44 |
| gan_patchgan | 0.0077 | 0.0002 |
|
| 45 |
| **gan_dinov2** | **0.0235** | **0.0030** |
|
| 46 |
| gan_dinov2_text | 0.0152 | 0.0012 |
|
| 47 |
+
| gan_dinov2_t035 | 0.0083 | 0.0001 |
|
| 48 |
| gan_dinov2_50 | 0.0038 | 0.0000 |
|
| 49 |
|
| 50 |
gan_dinov2 most realistic here (β3Γ others), then gan_dinov2_text; absolute values tiny (SVM still
|
| 51 |
separates real vs gen ~98%+). Precision (dropped from main table) @25k: patchgan 0.407 > baseline 0.402 >
|
| 52 |
+
original 0.380 > gan_dinov2 0.351 > t035 0.334 > dinov2_50 0.323 > drift(15k) 0.301 β inverse of realism (P/R tradeoff).
|