| --- |
| license: mit |
| tags: |
| - deepfake-detection |
| - face-forgery |
| - fourier-transform |
| - steganalysis |
| - srm |
| - computer-vision |
| - pytorch |
| --- |
| |
| # MFFI: Model Checkpoints & Benchmarks for Face Forgery Detection |
|
|
| Public weights, representations, and experiment artifacts for the project: |
| > **Benchmarking Spatial, Spectral, and Self-Supervised Cues for Face Forgery Detection under Realistic Degradation** |
| > 🔗 Official Codebase: [github.com/lucasdocunha/tcc](https://github.com/lucasdocunha/tcc) |
|
|
| --- |
|
|
| ## 📌 Overview |
|
|
| This repository hosts official pre-trained and fine-tuned model checkpoints (`best.pth`), evaluation metrics, and experiment configurations evaluated across **FaceForensics++ (clean and test_d)**, **Celeb-DF v2**, and the **DF-40 (DeepFake-40)** benchmarks. |
| |
| ### Included Model Families: |
| - **CLIP-ViT-B/16** (Multimodal Vision Transformer, OpenAI) |
| - **DINO (ConvNeXt-Base)** (Self-supervised distillation via DINOv3 LVD-1689M) |
| - **ViT-B/16** (Vision Transformer, ImageNet-21k) |
| - **Xception** (Depthwise Separable CNN) |
| - **ResNet-18** (Residual CNN) |
| - **MobileNetV3-Large** (Efficient Mobile CNN with Squeeze-and-Excitation) |
| - **Mixture of Experts (MoE)** (Standard and Frequency-specialized MoE architectures) |
| |
| ### Feature Representations & Frequency Modes: |
| - `none`: Spatial domain RGB (3 channels) |
| - `srm`: **Steganalysis Rich Models (SRM)** high-pass residual filters (3 channels: horizontal, vertical, diagonal edge residuals) |
| - `magnitude`: FFT Log-Magnitude $\log(|F| + 1)$ (1 channel) |
| - `phase`: FFT Phase angle $[0, 1]$ (1 channel) |
| - `complex`: Real and Imaginary components (2 channels) |
| - `concat`: Spatial RGB concatenated with FFT Magnitude (4 channels) |
| - `frequency_3`: High-pass filtered magnitude (1 channel) |
| - `concat_frequency`: Multi-domain RGB + Magnitude + Phase + HighPass + LowPass (7 channels) |
| |
| --- |
| |
| ## 🏆 Benchmark Highlights: SRM & Top Ensembles |
| |
| ### 1. Top Ensemble (CLIP-SRM + DINO-SRM) |
| Evaluating across 5 statistical seeds (`42`, `123`, `2024`, `7`, `2025`): |
| - **FF++ Test Limpo:** **94.17% AUC** (Mega 10x) / **93.07% ± 0.61%** (Ensemble 2x) |
| - **FF++ Test-D (Difícil):** **88.60% AUC** (Mega 10x) / **87.24% ± 0.49%** (Ensemble 2x) |
| - **DF-40 (40 Generative Methods):** **84.15% AUC** (Mega 10x) / **83.53% ± 1.74%** (Ensemble 2x) |
| - **Celeb-DF v2:** **75.06% AUC** (Mega 10x) / **74.11% ± 3.14%** (Ensemble 2x) |
| |
| ### 2. DF-40 Breakdown by Paradigm: |
| - **Editing (T2I & Inversion):** 97.47% Recall |
| - **Face Swap:** 93.30% Recall |
| - **Diffusion (SD 2.1, PixArt, DiT):** 93.32% Recall (SD 2.1: 100%, PixArt: 99.6%) |
| - **GANs (StyleGAN 2/3, VQGAN):** 90.96% Recall (StyleGAN 2/3: 100%) |
| - **Commercial Avatars & Talking Heads:** 68.53% AUC (HeyGen: 33.2% recall) |
| |
| --- |
| |
| ## 📂 Repository Structure |
| |
| ```text |
| ├── <model_family>/ |
| │ └── srm/ |
| │ └── finetune_robust/ |
| │ └── seed_<seed>/ |
| │ ├── weights/best.pth # Best checkpoint saved by validation ROC-AUC |
| │ ├── results/metrics_*.csv # Full metrics across test, test_d, df40, celeb_df |
| │ ├── results/run_config.json# Hyperparameters and seed configuration |
| │ └── plots/*.png # Confusion matrix and ROC-AUC plots |
| ├── tables/ # Consolidated CSV benchmark tables |
| │ ├── tabela9_srm_5seeds.csv |
| │ ├── df40_srm_breakdown_by_paradigm.csv |
| │ └── df40_srm_breakdown_by_technique.csv |
| └── figures/ # Multimodal attention rollout maps |
| └── heatmap_clip_dino_fusion_4rows.png |
| ``` |
| |
| --- |
| |
| ## 🚀 How to Download & Load Weights |
| |
| ```python |
| import torch |
| from huggingface_hub import hf_hub_download |
| |
| # Download best.pth for CLIP (SRM mode, finetune_robust, seed 42) |
| checkpoint_path = hf_hub_download( |
| repo_id="lucasoc/MFFI-Models", |
| filename="clip/srm/finetune_robust/seed_42/weights/best.pth" |
| ) |
| |
| # Load state dict |
| state_dict = torch.load(checkpoint_path, map_location="cpu", weights_only=True) |
| print(f"Loaded {len(state_dict)} tensors successfully!") |
| ``` |
| |
| --- |
| |
| ## 📄 Codebase & Citation |
| |
| For reproduction scripts, training configurations, and data preparation pipelines: |
| - **Repository:** [https://github.com/lucasdocunha/tcc](https://github.com/lucasdocunha/tcc) |
| - **License:** MIT License |
| |
