Spaces:
Running on Zero
Running on Zero
|
Download README.md from Thiru1122/Multimodel_setup: direct link, hf CLI and curl.
- Browser
- Download file 5.64 kB
-
https://huggingface.co/spaces/Thiru1122/Multimodel_setup/resolve/main/README.md
- Command line
-
hf download hf://spaces/Thiru1122/Multimodel_setup/README.md
-
curl -L -o README.md https://huggingface.co/spaces/Thiru1122/Multimodel_setup/resolve/main/README.md
5.64 kB
A newer version of the Gradio SDK is available: 6.32.0
metadata
title: Multimodel Setup
emoji: π¦
colorFrom: pink
colorTo: red
sdk: gradio
sdk_version: 5.20.0
python_version: '3.12'
app_file: app.py
pinned: false
Multi-Model Deepfake Detection & Cross-Generator Evaluation
A multi-model deepfake detection framework that combines specialized computer vision architectures to detect AI-generated and manipulated images across diverse generator families (GANs, Diffusion, Midjourney, StyleGAN, ADM).
π‘ The Core Idea
Single deepfake detectors suffer from generator overfitting:
- A model trained on facial GANs (like StyleGAN) often fails on diffusion models (like Midjourney).
- A model trained on general diffusion artifacts might miss fine-grained facial manipulations.
Our Solution: Instead of relying on a single fallible detector, we run multiple complementary vision models with distinct inductive biases and combine their outputs using deterministic decision fusion.
flowchart TD
IMG["Input Image"] --> P1["ViT Preprocessor (224x224 Resize)"]
IMG --> P2["UnivFD Preprocessor (Native CenterCrop + CLIP Norm)"]
IMG --> P3["ConvNeXt Preprocessor (Native CenterCrop + ImageNet Norm)"]
P1 --> M1["ViT Detector<br/>(Facial & GAN Artifacts)"]
P2 --> M2["UnivFD<br/>(CLIP ViT-L/14 + Linear Probe)"]
P3 --> M3["ConvNeXt DRCT<br/>(Diffusion Contrastive Base)"]
M1 -->|Fake Prob / Vote| FUSION{"Decision Fusion Engine<br/>(src/decision_fusion.py)"}
M2 -->|Fake Prob / Vote| FUSION
M3 -->|Fake Prob / Vote| FUSION
FUSION --> DECISION["Final Output:<br/>β’ Prediction: REAL / FAKE<br/>β’ Fused Confidence Score<br/>β’ Per-Model Explanation"]
π The Base Detectors
| Model | Architecture | Specialization / Training | Key Strength |
|---|---|---|---|
| ViT | Vision Transformer (dima806) |
Facial deepfakes & GAN faces | High sensitivity on facial boundaries and warping |
| UnivFD | CLIP ViT-L/14 + Linear Classifier | Multi-generator frequency & semantic cues | Universal zero-shot artifact sensitivity |
| ConvNeXt DRCT | ConvNeXt-Base (CVPR 2024) | Diffusion Reconstruction Contrastive Training | Outstanding generalization to diffusion (100% on Midjourney) |
βοΈ How It Works: Inference & Fusion Flow
sequenceDiagram
autonumber
actor User
participant App as Web UI / API (app.py)
participant Reg as Detector Registry
participant Models as Base Detectors (ViT, UnivFD, ConvNeXt)
participant Fusion as Decision Fusion
User->>App: Upload Image
App->>Reg: Request Predictions
par Run Detectors in Parallel / Safe Sequential
Reg->>Models: ViT Inference
Reg->>Models: UnivFD Inference
Reg->>Models: ConvNeXt DRCT Inference
end
Models-->>Reg: Individual Probabilities & Votes
Reg->>Fusion: Submit {ViT: p1, UnivFD: p2, ConvNeXt: p3}
Note over Fusion: Apply Strategy (Majority Vote / Mean Prob / Priority)
Fusion-->>App: Unified Fused Verdict + Confidence Breakdown
App-->>User: Display Results, Charts & Confidence
Fusion Strategies
- Majority Voting: Collects discrete votes (
REALvsFAKE). If a tie occurs, falls back to probability thresholding. - Mean Fake Probability: Computes the unweighted average of calibrated fake probabilities across all active detectors.
- Domain Priority: Selects primary detector based on input context (e.g. facial vs non-facial diffusion synthesis).
- Graceful Degradation: If any detector fails or checkpoint is absent, the engine falls back gracefully without crashing.
π Evaluation & Parity Verification
Our framework was evaluated against both facial GAN benchmarks and external diffusion datasets (Midjourney v5, ADM, ImageNet nature photos):
flowchart LR
A["Test Datasets"] --> B["Phase 6: FFHQ vs StyleGAN (Facial)"]
A --> C["Phase 7: Midjourney + ADM vs ImageNet (Diffusion)"]
B --> D["ViT: 99.5% Acc<br/>ConvNeXt: 82.5% Acc"]
C --> E["ConvNeXt DRCT: 77.0% Acc (100% Midjourney Recall)<br/>UnivFD: 57.5% Acc<br/>ViT: 52.0% Acc"]
- Reference Parity: All adapters match official paper implementations (CVPR 2023 UniversalFakeDetect and CVPR 2024 DRCT) with numerical differences $< 10^{-6}$.
- Comprehensive Regression Tests: Automated test suite with 25 unit and contract tests verifying preprocessing shapes, normalization, and fusion edge cases.
π Quickstart
1. Installation
git clone https://github.com/Thiru-Selvam-06/Deepfake-Detection.git
cd Deepfake-Detection
pip install -r requirements.txt
2. Run the Interactive Web UI
python app.py
Open your browser at http://localhost:7860.
3. Run Automated Tests
pytest tests/ -v
π Repository Structure
βββ app.py # Gradio Web Interface
βββ src/
β βββ convnext_adapter.py # DRCT Native Crop Preprocessing & Inference
β βββ univfd_adapter.py # UniversalFakeDetect CLIP Preprocessing & Inference
β βββ vit_adapter.py # Vision Transformer Adapter
β βββ decision_fusion.py # Multi-Model Voting & Probability Fusion Logic
β βββ detector_registry.py # Unified Lifecycle & Model Registry
βββ benchmark/ # Labeled test manifests and evaluation results
βββ tests/ # Regression and unit tests for adapters and fusion
βββ docs/ # Detailed scientific evaluation reports
βββ models/ # Pretrained model weights and checkpoints (Git LFS)