Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
SynthScope Leaderboard (v1)
Auto-generated by benchmark/leaderboard.py — do not edit by hand. See benchmark/README.md for the protocol and how to submit.
This is the standardized, versioned table: a few tracks with a fixed protocol that anyone can submit to. The web page's Leaderboard tab is broader (every paper benchmark, 30+ groups) and the repo supports many more datasets, which download and organize themselves: see docs/BENCHMARKS.md.
Metrics are percentages. mAP = mean AP over generators. Each track is ranked by its primary metric (in bold).
ForenSynths — image (ProGAN-trained, cross-generator)
| Rank | Method | Eval | Acc | mAP | AP | AUROC | Verified | Notes |
|---|---|---|---|---|---|---|---|---|
| 1 | FatFormer | official-weights | 98.39 | 99.70 | — | — | ✓ | ForenSynths-8 subset; official released weights |
| 2 | CO-SPY (official weights) | official-weights | 95.12 | 98.71 | — | — | ✓ | ForenSynths-8; vendored official architecture |
| 3 | UnivFD / CLIP | frozen-probe | — | 97.20 | — | 97.00 | ✓ | ForenSynths; frozen CLIP ViT-L/14 probe (ProGAN) |
| 4 | CO-SPY (from scratch) | from-scratch | — | 96.98 | — | 96.79 | ✓ | ForenSynths; SynthScope re-implementation trained on ProGAN |
| 5 | CNNDetection | from-scratch | — | 96.80 | — | 96.60 | ✓ | ForenSynths; ResNet-50 fine-tune (ProGAN) |
| 6 | DINOv2 probe | frozen-probe | — | 95.03 | — | 94.86 | ✓ | ForenSynths; frozen DINOv2 ViT-L/14 probe |
| 7 | SAFE | from-scratch | — | 94.13 | — | 93.68 | ✓ | ForenSynths; trained from scratch |
| 8 | AIDE | official-weights | — | 92.08 | — | — | ✓ | ForenSynths-8; mAP reproduces, acc@0.5 miscalibrated (single-view vs official 5-view DCT) |
| 9 | NPR | official-weights | 91.83 | 87.09 | — | — | ✓ | ForenSynths-8; official ProGAN weights |
GenImage SD-v1.4 — image
| Rank | Method | Eval | Acc | mAP | AP | AUROC | Verified | Notes |
|---|---|---|---|---|---|---|---|---|
| 1 | VIB-Net | from-scratch | — | 99.33 | — | 99.33 | ✓ | GenImage SD-v1.4 single real/fake pool (mAP == AUROC by construction) |
| 2 | SPAI (official weights) | official-weights | — | — | 94.01 | 98.45 | ✓ | GenImage SD-v1.4; vendored official architecture, fixed-res 2x2x224 |
| 3 | SPAI (from scratch) | from-scratch | — | — | — | 97.66 | ✓ | DMID Latent-Diffusion vs COCO; SynthScope fixed-resolution variant |
GenVideo-Val — video
| Rank | Method | Eval | Acc | mAP | AP | AUROC | Verified | Notes |
|---|---|---|---|---|---|---|---|---|
| 1 | ReStraV | frozen-probe | — | — | 96.67 | 97.06 | ✓ | Frozen DINOv2 trajectory geometry + MLP |
| 2 | D3 | training-free | — | — | 95.60 | 93.60 | ✓ | Training-free frozen CLIP 2nd-order trajectory |