husseinelsaadi/aidetect-vit-b16

AI Image Detector — ViT-B/16 (CLIP-initialised)

Binary classifier: real photograph vs AI-generated image. Fine-tuned from the openai/clip-vit-base-patch16 vision tower on ~195,000 images spanning 1,100+ distinct generators, and — the part that usually goes unreported — measured on generators it never saw during training.

🔍 Try it in the browser

Why another detector

Published AI-image detectors routinely report 99% and then collapse on anything their training generator did not produce. Wang et al. (2020) documented it; Bird & Lotfi (2024) hit 92.98% on CIFAKE using a single generator and named that as their central limitation. This model is an attempt to answer the question their limitation raises: how far does accuracy actually fall on generators nobody trained against?

The answer here is 97.6% → ~89.8%, and the failures are not where intuition says.

Results

Evaluation set	What it measures	Accuracy	AUROC
Held-out validation	seen generators	0.976	0.997
OpenFake core/test	unseen generators + unseen real sources	0.898	0.960
OpenFake reddit/test	in-the-wild Reddit uploads	0.869	0.939
CIFAKE	Bird & Lotfi (2024) benchmark, 32×32	0.720	0.815

Two notes on reading this table honestly:

• The 0.898 is at the deployed threshold of 0.71, which was fitted on one half of the unseen-generator split and reported on the held-out half. At the threshold fitted on in-distribution validation instead, the same model scores 0.881 there. The threshold is part of the model; both numbers are given so neither looks cherry-picked.
• reddit/test is 83% fake, so read its AUROC rather than its accuracy — a model that simply answers "always AI" scores 82.6% on it.

Architecture comparison

A ResNet-50 trained on the identical data (2 epochs, fixed budget) reaches 0.945 in-distribution but only 0.813 / AUROC 0.896 on unseen generators. The gap between the two architectures triples once the generator is unfamiliar — which is the argument for CLIP initialisation, arriving as a measurement rather than a claim.

Unseen generators are caught reliably. On core/test: gpt-image-1.5 0.977, z-image-turbo 0.960, flux.2-klein-9b 0.930, midjourney-7 0.911 — none of them in training. The residual error is dominated by the other direction: genuine photographs from unfamiliar camera pipelines (DOCCI, ImageNet) being called AI. Full-range resolution augmentation cut that false-positive rate from ~25% to ~9%.

What is in this repository

Each run changes exactly one thing, so any difference between them is attributable. runC is the model served by the demo and the one these numbers describe.

Folder	Training recipe	Unseen-generator accuracy
runA/	baseline augmentation (crop + flip)	0.842
runB/	+ JPEG q30–100, blur, mild rescale	0.856
runC/	+ full-range resolution collapse — shipped	0.881
runD/	runC + simulated smartphone pipeline	0.864 — did not help

runD is kept because a negative result is still a result: adding a simulated phone camera pipeline made the detector worse at catching fakes and worse at leaving real photographs alone. Each folder holds checkpoints/best.pt, checkpoints/last.pt and a summary.json with the full training history and calibration.

Usage

import torch, json
from huggingface_hub import hf_hub_download
from PIL import Image

# preprocessing must match training exactly — see the code repo's preprocess.py
ckpt = hf_hub_download("husseinelsaadi/aidetect-vit-b16", "runC/checkpoints/best.pt")
blob = torch.load(ckpt, map_location="cpu", weights_only=False)

# blob["model"]  -> state dict
# blob["extra"]["config"] -> the training configuration

The complete, runnable inference path — canonical preprocessing, whole-frame tiling, temperature scaling and the tuned threshold — is the Space's app.py. Copy it rather than reimplementing: a preprocessing mismatch silently costs several points of accuracy.

Decision rule. Apply temperature scaling (summary.json → temperature), then threshold at the published operating point rather than 0.5. The threshold was fitted on held-out out-of-distribution data, because on a live demo the expensive mistake is accusing a real photograph, not missing one synthetic image.

Training

Backbone	openai/clip-vit-base-patch16 vision tower, last 4 blocks + head trainable (~28M params)
Data	~195k images, 50/50 real/fake, 1,118 generators, capped per generator
Preprocessing	short side → 320 (LANCZOS), centre crop 256, JPEG q95 re-encode for both classes
Augmentation	RandomResizedCrop(224, 0.6–1.0), flip, JPEG q30–100, blur σ0–3, full-range down/upscale
Optimiser	AdamW, wd 0.05, head LR 1e-3, backbone 1e-5, layer-wise decay 0.75, 5% warmup → cosine
Precision / hardware	bf16 on one NVIDIA L4
Loss	BCE-with-logits, label smoothing 0.05

The control that makes the numbers meaningful

In the source data, fakes are PNGs straight from the generator and reals are JPEGs from photo datasets. Handed that raw, a model scores ~99% by learning "PNG means fake" and has learned nothing about AI. Every image of both classes therefore passes through one re-encoding function with no branch on the label.

The check that this worked: training the same configuration on randomly permuted labels gives 0.511 accuracy — chance. The classes are not separable by format, resolution or compression, so the reported numbers reflect generated-image structure rather than a pipeline artifact.

Limitations and intended use

• Evidence, not proof. Academic coursework, not a forensic instrument. Do not use it to make consequential accusations about a person or a publication.
• About 1 in 11 genuine photographs from unfamiliar camera pipelines is still flagged as AI. Unfamiliar sensor and compression statistics resemble generation.
• Low-resolution images are harder: CIFAKE at 32×32 scores 0.720 against 0.976 in-distribution. Heavy upscaling is its own distribution shift.
• Screenshots, memes and heavily edited photographs sit between the two classes.
• Generators released after training will drift away from what was measured here.
• Non-commercial use only — the training data is CC-BY-NC-SA / CC-BY-NC.

Citation

@misc{elsaadi2026aidetect,
  author = {El Saadi, Hussein},
  title  = {AI Image Detector: CLIP-ViT-B/16 trained across 1,100+ generators},
  year   = {2026},
  note   = {CSC625 Deep Learning, Modern University for Business and Science (MUBS)},
  url    = {https://huggingface.co/husseinelsaadi/aidetect-vit-b16}
}

References

• Bird & Lotfi (2024), CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images, IEEE Access 12:15642–15650
• Wang et al. (2020), CNN-Generated Images Are Surprisingly Easy to Spot… For Now, CVPR
• Ojha et al. (2023), Towards Universal Fake Image Detectors that Generalize Across Generative Models, CVPR
• Park & Owens (2025), Community Forensics: Using Thousands of Generators to Train Fake Image Detectors, CVPR
• Livernoche et al. (2025), OpenFake: An Open Dataset and Platform Toward Real-World Deepfake Detection, arXiv:2509.09495

---

CSC625 Deep Learning · Hussein El Saadi · Modern University for Business and Science (MUBS) · Summer 2026.