README.md

---
license: mit
tags:
- deepfake-detection
- face-forgery
- fourier-transform
- steganalysis
- srm
- computer-vision
- pytorch
---

# MFFI: Model Checkpoints & Benchmarks for Face Forgery Detection

Public weights, representations, and experiment artifacts for the project:
> **Benchmarking Spatial, Spectral, and Self-Supervised Cues for Face Forgery Detection under Realistic Degradation**  
> 🔗 Official Codebase: [github.com/lucasdocunha/tcc](https://github.com/lucasdocunha/tcc)

---

## 📌 Overview

This repository hosts official pre-trained and fine-tuned model checkpoints (`best.pth`), evaluation metrics, and experiment configurations evaluated across **FaceForensics++ (clean and test_d)**, **Celeb-DF v2**, and the **DF-40 (DeepFake-40)** benchmarks.

### Included Model Families:
- **CLIP-ViT-B/16** (Multimodal Vision Transformer, OpenAI)
- **DINO (ConvNeXt-Base)** (Self-supervised distillation via DINOv3 LVD-1689M)
- **ViT-B/16** (Vision Transformer, ImageNet-21k)
- **Xception** (Depthwise Separable CNN)
- **ResNet-18** (Residual CNN)
- **MobileNetV3-Large** (Efficient Mobile CNN with Squeeze-and-Excitation)
- **Mixture of Experts (MoE)** (Standard and Frequency-specialized MoE architectures)

### Feature Representations & Frequency Modes:
- `none`: Spatial domain RGB (3 channels)
- `srm`: **Steganalysis Rich Models (SRM)** high-pass residual filters (3 channels: horizontal, vertical, diagonal edge residuals)
- `magnitude`: FFT Log-Magnitude $\log(|F| + 1)$ (1 channel)
- `phase`: FFT Phase angle $[0, 1]$ (1 channel)
- `complex`: Real and Imaginary components (2 channels)
- `concat`: Spatial RGB concatenated with FFT Magnitude (4 channels)
- `frequency_3`: High-pass filtered magnitude (1 channel)
- `concat_frequency`: Multi-domain RGB + Magnitude + Phase + HighPass + LowPass (7 channels)

---

## 🏆 Benchmark Highlights: SRM & Top Ensembles

### 1. Top Ensemble (CLIP-SRM + DINO-SRM)
Evaluating across 5 statistical seeds (`42`, `123`, `2024`, `7`, `2025`):
- **FF++ Test Limpo:** **94.17% AUC** (Mega 10x) / **93.07% ± 0.61%** (Ensemble 2x)
- **FF++ Test-D (Difícil):** **88.60% AUC** (Mega 10x) / **87.24% ± 0.49%** (Ensemble 2x)
- **DF-40 (40 Generative Methods):** **84.15% AUC** (Mega 10x) / **83.53% ± 1.74%** (Ensemble 2x)
- **Celeb-DF v2:** **75.06% AUC** (Mega 10x) / **74.11% ± 3.14%** (Ensemble 2x)

### 2. DF-40 Breakdown by Paradigm:
- **Editing (T2I & Inversion):** 97.47% Recall
- **Face Swap:** 93.30% Recall
- **Diffusion (SD 2.1, PixArt, DiT):** 93.32% Recall (SD 2.1: 100%, PixArt: 99.6%)
- **GANs (StyleGAN 2/3, VQGAN):** 90.96% Recall (StyleGAN 2/3: 100%)
- **Commercial Avatars & Talking Heads:** 68.53% AUC (HeyGen: 33.2% recall)

---

## 📂 Repository Structure

```text
├── <model_family>/
│   └── srm/
│       └── finetune_robust/
│           └── seed_<seed>/
│               ├── weights/best.pth       # Best checkpoint saved by validation ROC-AUC
│               ├── results/metrics_*.csv  # Full metrics across test, test_d, df40, celeb_df
│               ├── results/run_config.json# Hyperparameters and seed configuration
│               └── plots/*.png            # Confusion matrix and ROC-AUC plots
├── tables/                                # Consolidated CSV benchmark tables
│   ├── tabela9_srm_5seeds.csv
│   ├── df40_srm_breakdown_by_paradigm.csv
│   └── df40_srm_breakdown_by_technique.csv
└── figures/                               # Multimodal attention rollout maps
    └── heatmap_clip_dino_fusion_4rows.png
```

---

## 🚀 How to Download & Load Weights

```python
import torch
from huggingface_hub import hf_hub_download

# Download best.pth for CLIP (SRM mode, finetune_robust, seed 42)
checkpoint_path = hf_hub_download(
    repo_id="lucasoc/MFFI-Models",
    filename="clip/srm/finetune_robust/seed_42/weights/best.pth"
)

# Load state dict
state_dict = torch.load(checkpoint_path, map_location="cpu", weights_only=True)
print(f"Loaded {len(state_dict)} tensors successfully!")
```

---

## 📄 Codebase & Citation

For reproduction scripts, training configurations, and data preparation pipelines:
- **Repository:** [https://github.com/lucasdocunha/tcc](https://github.com/lucasdocunha/tcc)
- **License:** MIT License