File size: 4,424 Bytes
e6a5830
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
---
license: other
license_name: dinov3-license
license_link: https://ai.meta.com/resources/models-and-libraries/dinov3-license/
library_name: vae-rhythm
pipeline_tag: video-classification
tags:
  - ai-generated-video-detection
  - synthetic-video-detection
  - video-forensics
  - dinov3
base_model: facebook/dinov3-vitl16-pretrain-lvd1689m
---

# CaDet: AI-generated video detector (VAE Rhythm)

<p align="center"><img src="https://raw.githubusercontent.com/showlab/vae-rhythm/main/assets/teaser.webp" width="70%" alt="A 4x causal VAE decodes frames in groups, and the phase scan reads the resulting periodic trace"></p>

CaDet is the detector of the paper **VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection**
([paper](https://arxiv.org/abs/2611.21114), [code](https://github.com/showlab/vae-rhythm), [project page](https://showlab.github.io/vae-rhythm/)).

It attaches two parameter-efficient paths to one frozen **DINOv3 ViT-L/16** and reads 8 frames per clip:

* **Artifact path** — LoRA (rank 16) on the attention qkv projections, trained on AIGVDBench plus **VAE pairs**
  (real clips and copies that differ only by one video-VAE round trip). One forward pass feeds a generic head
  (both pair members counted real) and an artifact head on blocks 8 / 12 / 16 (VAE copies counted generated).
* **Semantic path** — Effort-style tuning of the smallest singular component of each of the 96 attention
  projections plus a 2-way head, trained on AIGVDBench only.

The video score is the sum of both paths after standardising each on AIGVDBench validation real videos.
It is a z-score sum (≈ 0 for typical real videos, larger = more likely generated), **not a probability**.

## Files

```
seed0/ seed1/ seed2/          three training seeds (the paper reports their mean)
  artifact_path.safetensors   LoRA adapters + both heads (≈ 16 MB)
  semantic_path.safetensors   rank-one SVD residuals + 2-way head (≈ 0.8 MB)
  calibration.json            head and fusion statistics from AIGVDBench validation reals
```

The DINOv3 backbone weights are not included; `timm` downloads them (`vit_large_patch16_dinov3.lvd1689m`)
after you accept the DINOv3 license on Hugging Face.

## Usage

```bash
pip install "vae-rhythm[cadet] @ git+https://github.com/showlab/vae-rhythm"
hf download Jack97/CaDet --local-dir checkpoints/cadet
vae-rhythm predict my_video.mp4 --weights checkpoints/cadet/seed0
```

```python
from vae_rhythm.cadet.detector import CaDet
det = CaDet("checkpoints/cadet/seed0")          # needs a CUDA GPU (about 3 GB)
print(det.score_video("my_video.mp4"))           # {'score': ..., 'z_semantic': ..., 'z_artifact': ...}
```

Frames 0, 4, …, 28 of the clip are centre-cropped to a square and resized to 256 × 256 (JPEG q95), as in
training. Clips shorter than 29 frames are padded by repeating frames uniformly.

## Results (AUROC %, mean of 3 seeds)

Trained only on AIGVDBench's Open-Sora and real clips (plus VAE pairs for the artifact path); all other
generators are unseen.

| AIGVDBench 20 open | 19 unseen open | 11 closed | 2026 closed (MiniMax H3, Seedance 2.0 / 2.5) | DF26 all | DF26 open | DF26 closed | DF26 paired 2AFC |
|---|---|---|---|---|---|---|---|
| 93.9 | 93.6 | 95.8 | 88.3 | 78.5 | 78.2 | 79.0 | 87.8 |

Adding the training-free VAE Rhythm plug-in (phase scan, see the code repository) raises DF26 further.

## Intended use and limitations

Research on AI-generated video detection and as one piece of evidence in media verification. Do not use the
score alone to make consequential decisions about a person or a video.

* Operating points must be chosen on real videos from the deployment domain; the score distribution of real
  videos shifts across sources (in the paper, swapping the real source lowers AUROC for every trained detector).
* Heavily re-encoded, very short, very low-resolution or edited videos are harder; performance on generators
  released after the training data is not guaranteed.
* Trained on AIGVDBench (real clips from OpenVid) — content outside that distribution (e.g. talking-head
  livestreams, screen recordings) has not been characterised.

## Citation

```bibtex
@article{sun2026vaerhythm,
  title   = {VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection},
  author  = {Sun, Ke and Li, Shuwei and Mei, Haiyang and Shou, Mike Zheng},
  journal = {arXiv preprint arXiv:2611.21114},
  year    = {2026}
}
```