DF26

DF26 is an evaluation set for video-level (temporal) and frame-based deepfake detectors, covering AI-generated talking-head video: one speaker, in casual address, studio interview, or official statement. Each real clip has a single scene prompt used by every generator, so identity and scene stay fixed and the generator is what changes. Paper: arXiv:2609.07369.

There are 271 reals and 2,420 fakes, stored as 1280×720 H.264. Duration is targeted at 5 s (4.01–6.04s in reality: Grok Imagine 6.04 s, most Veo 3.1 clips 4.01 s, the rest ≈5 s). metadata/videos.csv has one row per video.

Real clips are under real/{scenario}/ and come from OpenVid-1M, MAVOS-DD, and TalkingCelebs:

Fakes are fake/{scenario}/{generator}/{clip_id}.mp4 with the same clip_id as the real. Open-weight models provide I2V (real frame + prompt) and T2V (prompt only). Commercial APIs are I2V only and were not run on Official_Statement.

real/{scenario}/{clip_id}.mp4
fake/{scenario}/{generator}/{clip_id}.mp4
frames/{scenario}/{clip_id}/{clip_id}_frame{0-3}.jpg
prompts.csv
metadata/videos.csv
metadata/frames.csv
metadata/generators.csv
dataset-creation-and-evaluation/

clip_id joins the real, its fakes, four keyframes, and the prompts.csv row ({scenario}/{clip_id}.mp4). Paths in the metadata CSVs are relative to the repo root.

Baselines

Cross-generator video-level AUROC (%) of state-of-the-art deepfake detectors trained on FF++ and evaluated on DF26 without fine-tuning (Table 6 of the paper). Mean is macro-averaged over the ten generator subsets. Higher is better; 50 is chance.

The first four columns are commercial generators, the rest open-weight. No detector is uniformly strong: PwTF-DVD reaches 92.9 AUROC on Wan 2.2 T2V yet is at chance on HunyuanVideo 1.5 I2V.

Evaluation

Report AUC and EER per generator. Training rules are in LICENSE.md.

Citation

@misc{shykula2026df26,
  title         = {DF26: We Cannot Tell Fake From Real Anymore},
  author        = {Shykula, Severyn and Yermakov, Andrii and Samarskyi, Ivan and Mishkin, Dmytro and Cech, Jan and Mishchuk, Anastasiia},
  year          = {2026},
  eprint        = {2609.07369},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2609.07369}
}