metadata
license: mit
tags:
- deepfake-detection
- face-forgery
- fourier-transform
- steganalysis
- srm
- computer-vision
- pytorch
MFFI: Model Checkpoints & Benchmarks for Face Forgery Detection
Public weights, representations, and experiment artifacts for the project:
Benchmarking Spatial, Spectral, and Self-Supervised Cues for Face Forgery Detection under Realistic Degradation
🔗 Official Codebase: github.com/lucasdocunha/tcc
📌 Overview
This repository hosts official pre-trained and fine-tuned model checkpoints (best.pth), evaluation metrics, and experiment configurations evaluated across FaceForensics++ (clean and test_d), Celeb-DF v2, and the DF-40 (DeepFake-40) benchmarks.
Included Model Families:
- CLIP-ViT-B/16 (Multimodal Vision Transformer, OpenAI)
- DINO (ConvNeXt-Base) (Self-supervised distillation via DINOv3 LVD-1689M)
- ViT-B/16 (Vision Transformer, ImageNet-21k)
- Xception (Depthwise Separable CNN)
- ResNet-18 (Residual CNN)
- MobileNetV3-Large (Efficient Mobile CNN with Squeeze-and-Excitation)
- Mixture of Experts (MoE) (Standard and Frequency-specialized MoE architectures)
Feature Representations & Frequency Modes:
none: Spatial domain RGB (3 channels)srm: Steganalysis Rich Models (SRM) high-pass residual filters (3 channels: horizontal, vertical, diagonal edge residuals)magnitude: FFT Log-Magnitude $\log(|F| + 1)$ (1 channel)phase: FFT Phase angle $[0, 1]$ (1 channel)complex: Real and Imaginary components (2 channels)concat: Spatial RGB concatenated with FFT Magnitude (4 channels)frequency_3: High-pass filtered magnitude (1 channel)concat_frequency: Multi-domain RGB + Magnitude + Phase + HighPass + LowPass (7 channels)
🏆 Benchmark Highlights: SRM & Top Ensembles
1. Top Ensemble (CLIP-SRM + DINO-SRM)
Evaluating across 5 statistical seeds (42, 123, 2024, 7, 2025):
- FF++ Test Limpo: 94.17% AUC (Mega 10x) / 93.07% ± 0.61% (Ensemble 2x)
- FF++ Test-D (Difícil): 88.60% AUC (Mega 10x) / 87.24% ± 0.49% (Ensemble 2x)
- DF-40 (40 Generative Methods): 84.15% AUC (Mega 10x) / 83.53% ± 1.74% (Ensemble 2x)
- Celeb-DF v2: 75.06% AUC (Mega 10x) / 74.11% ± 3.14% (Ensemble 2x)
2. DF-40 Breakdown by Paradigm:
- Editing (T2I & Inversion): 97.47% Recall
- Face Swap: 93.30% Recall
- Diffusion (SD 2.1, PixArt, DiT): 93.32% Recall (SD 2.1: 100%, PixArt: 99.6%)
- GANs (StyleGAN 2/3, VQGAN): 90.96% Recall (StyleGAN 2/3: 100%)
- Commercial Avatars & Talking Heads: 68.53% AUC (HeyGen: 33.2% recall)
📂 Repository Structure
├── <model_family>/
│ └── srm/
│ └── finetune_robust/
│ └── seed_<seed>/
│ ├── weights/best.pth # Best checkpoint saved by validation ROC-AUC
│ ├── results/metrics_*.csv # Full metrics across test, test_d, df40, celeb_df
│ ├── results/run_config.json# Hyperparameters and seed configuration
│ └── plots/*.png # Confusion matrix and ROC-AUC plots
├── tables/ # Consolidated CSV benchmark tables
│ ├── tabela9_srm_5seeds.csv
│ ├── df40_srm_breakdown_by_paradigm.csv
│ └── df40_srm_breakdown_by_technique.csv
└── figures/ # Multimodal attention rollout maps
└── heatmap_clip_dino_fusion_4rows.png
🚀 How to Download & Load Weights
import torch
from huggingface_hub import hf_hub_download
# Download best.pth for CLIP (SRM mode, finetune_robust, seed 42)
checkpoint_path = hf_hub_download(
repo_id="lucasoc/MFFI-Models",
filename="clip/srm/finetune_robust/seed_42/weights/best.pth"
)
# Load state dict
state_dict = torch.load(checkpoint_path, map_location="cpu", weights_only=True)
print(f"Loaded {len(state_dict)} tensors successfully!")
📄 Codebase & Citation
For reproduction scripts, training configurations, and data preparation pipelines:
- Repository: https://github.com/lucasdocunha/tcc
- License: MIT License