|
Download README.md from Jack97/CaDet: direct link, hf CLI and curl.
- Browser
- Download file 4.42 kB
-
https://huggingface.co/Jack97/CaDet/resolve/main/README.md
- Command line
-
hf download hf://Jack97/CaDet/README.md
-
curl -L -o README.md https://huggingface.co/Jack97/CaDet/resolve/main/README.md
4.42 kB
| license: other | |
| license_name: dinov3-license | |
| license_link: https://ai.meta.com/resources/models-and-libraries/dinov3-license/ | |
| library_name: vae-rhythm | |
| pipeline_tag: video-classification | |
| tags: | |
| - ai-generated-video-detection | |
| - synthetic-video-detection | |
| - video-forensics | |
| - dinov3 | |
| base_model: facebook/dinov3-vitl16-pretrain-lvd1689m | |
| # CaDet: AI-generated video detector (VAE Rhythm) | |
| <p align="center"><img src="https://raw.githubusercontent.com/showlab/vae-rhythm/main/assets/teaser.webp" width="70%" alt="A 4x causal VAE decodes frames in groups, and the phase scan reads the resulting periodic trace"></p> | |
| CaDet is the detector of the paper **VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection** | |
| ([paper](https://arxiv.org/abs/2611.21114), [code](https://github.com/showlab/vae-rhythm), [project page](https://showlab.github.io/vae-rhythm/)). | |
| It attaches two parameter-efficient paths to one frozen **DINOv3 ViT-L/16** and reads 8 frames per clip: | |
| * **Artifact path** — LoRA (rank 16) on the attention qkv projections, trained on AIGVDBench plus **VAE pairs** | |
| (real clips and copies that differ only by one video-VAE round trip). One forward pass feeds a generic head | |
| (both pair members counted real) and an artifact head on blocks 8 / 12 / 16 (VAE copies counted generated). | |
| * **Semantic path** — Effort-style tuning of the smallest singular component of each of the 96 attention | |
| projections plus a 2-way head, trained on AIGVDBench only. | |
| The video score is the sum of both paths after standardising each on AIGVDBench validation real videos. | |
| It is a z-score sum (≈ 0 for typical real videos, larger = more likely generated), **not a probability**. | |
| ## Files | |
| ``` | |
| seed0/ seed1/ seed2/ three training seeds (the paper reports their mean) | |
| artifact_path.safetensors LoRA adapters + both heads (≈ 16 MB) | |
| semantic_path.safetensors rank-one SVD residuals + 2-way head (≈ 0.8 MB) | |
| calibration.json head and fusion statistics from AIGVDBench validation reals | |
| ``` | |
| The DINOv3 backbone weights are not included; `timm` downloads them (`vit_large_patch16_dinov3.lvd1689m`) | |
| after you accept the DINOv3 license on Hugging Face. | |
| ## Usage | |
| ```bash | |
| pip install "vae-rhythm[cadet] @ git+https://github.com/showlab/vae-rhythm" | |
| hf download Jack97/CaDet --local-dir checkpoints/cadet | |
| vae-rhythm predict my_video.mp4 --weights checkpoints/cadet/seed0 | |
| ``` | |
| ```python | |
| from vae_rhythm.cadet.detector import CaDet | |
| det = CaDet("checkpoints/cadet/seed0") # needs a CUDA GPU (about 3 GB) | |
| print(det.score_video("my_video.mp4")) # {'score': ..., 'z_semantic': ..., 'z_artifact': ...} | |
| ``` | |
| Frames 0, 4, …, 28 of the clip are centre-cropped to a square and resized to 256 × 256 (JPEG q95), as in | |
| training. Clips shorter than 29 frames are padded by repeating frames uniformly. | |
| ## Results (AUROC %, mean of 3 seeds) | |
| Trained only on AIGVDBench's Open-Sora and real clips (plus VAE pairs for the artifact path); all other | |
| generators are unseen. | |
| | AIGVDBench 20 open | 19 unseen open | 11 closed | 2026 closed (MiniMax H3, Seedance 2.0 / 2.5) | DF26 all | DF26 open | DF26 closed | DF26 paired 2AFC | | |
| |---|---|---|---|---|---|---|---| | |
| | 93.9 | 93.6 | 95.8 | 88.3 | 78.5 | 78.2 | 79.0 | 87.8 | | |
| Adding the training-free VAE Rhythm plug-in (phase scan, see the code repository) raises DF26 further. | |
| ## Intended use and limitations | |
| Research on AI-generated video detection and as one piece of evidence in media verification. Do not use the | |
| score alone to make consequential decisions about a person or a video. | |
| * Operating points must be chosen on real videos from the deployment domain; the score distribution of real | |
| videos shifts across sources (in the paper, swapping the real source lowers AUROC for every trained detector). | |
| * Heavily re-encoded, very short, very low-resolution or edited videos are harder; performance on generators | |
| released after the training data is not guaranteed. | |
| * Trained on AIGVDBench (real clips from OpenVid) — content outside that distribution (e.g. talking-head | |
| livestreams, screen recordings) has not been characterised. | |
| ## Citation | |
| ```bibtex | |
| @article{sun2026vaerhythm, | |
| title = {VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection}, | |
| author = {Sun, Ke and Li, Shuwei and Mei, Haiyang and Shou, Mike Zheng}, | |
| journal = {arXiv preprint arXiv:2611.21114}, | |
| year = {2026} | |
| } | |
| ``` | |