---
language:
- en
license: mit
tags:
- deepfake-detection
- video-deepfake
- face-forensics
- computer-vision
- gend
- pytorch
datasets:
- faceforensics
pipeline_tag: image-classification
base_model: openai/clip-vit-large-patch14
---

# GenD_CLIP_L_14_FF

State-of-the-art Deepfake Detection model trained on **FaceForensics++ (FF++)** based on the **GenD** framework fine-tuned with **Sharpness-Aware Minimization (SAM)** and **Label Smoothing**.

This model uses **openai/clip-vit-large-patch14** as visual foundation backbone and fine-tunes only the **Layer Normalization parameters** (accounting for only ~0.03% of total parameters) while enforcing a **hyperspherical feature manifold** through L2-normalization and metric learning (Uniformity & Alignment losses).


## 📊 Benchmark Results

| Metric | Score |
|:---|:---:|
| Video AUROC | **94.26%** |
| Video mAP | **93.30%** |
| Video Accuracy | **87.56%** |
| Video EER | **12.44%** |
| Frame AUROC | **90.57%** |
| Frame mAP | **89.14%** |
| Frame Accuracy | **82.91%** |

## 📌 Model Details

- **Training Dataset:** `FaceForensics++ (FF++)` (`FF++`)
- **Visual Backbone:** `openai/clip-vit-large-patch14`
- **Classification Head:** `LinearNorm`
- **Optimizer:** `SAM-AdamW` (SAM $\rho=0.05$, adaptive=True)
- **Loss Formulation:** Cross-Entropy with Label Smoothing (0.1), Uniformity (0.5), Alignment (0.1)
- **Training Epochs:** `30`
- **Batch Size:** `96`
- **Precision:** `bf16-mixed`
- **Learning Rate:** `0.0003`

## 🚀 Quickstart & Inference

### 1. Using the Model in Python

```python
import torch
from PIL import Image
from transformers import AutoModel

# Load the model directly from Hugging Face Hub
model = AutoModel.from_pretrained("Ant-One/GenD_CLIP_L_14_FF", trust_remote_code=True)
model.eval()

# Preprocess image crop (aligned face)
image = Image.open("path/to/face_crop.png").convert("RGB")
tensor = model.feature_extractor.preprocess(image).unsqueeze(0)

# Run inference
with torch.no_grad():
    logits = model(tensor)
    # Output class 0: Real, Output class 1: Fake
    fake_prob = logits.softmax(dim=-1)[0, 1].item()

print(f"Deepfake Probability: {fake_prob:.2%}")
```

### 2. Using with `src.hf.modeling_gend`

```python
from src.hf.modeling_gend import GenD

model = GenD.from_pretrained("Ant-One/GenD_CLIP_L_14_FF")
model.eval()
```

## 🏗️ Architecture & Training Methodology

The **GenD** method achieves superior cross-dataset generalization by avoiding catastrophic overfitting on manipulation-specific artifacts:
1. **Training on FF++:** Trained on face crops from FaceForensics++ (FF++).
2. **LayerNorm Tuning:** Keeps the visual transformer backbone frozen while updating only normalization scaling and bias terms.
3. **Normalized Linear Head:** Normalizes feature vectors onto a hypersphere before linear projection.
4. **SAM Optimization:** Smooths the loss landscape to find flat minima that resist out-of-distribution domain shifts.