Thiru1122 commited on
Commit
ebaaf8b
Β·
1 Parent(s): 151f388

docs: add concise README with architecture flowcharts and detector overview

Browse files
Files changed (1) hide show
  1. README.md +144 -13
README.md CHANGED
@@ -1,13 +1,144 @@
1
- ---
2
- title: Multimodel Setup
3
- emoji: πŸ¦€
4
- colorFrom: pink
5
- colorTo: red
6
- sdk: gradio
7
- sdk_version: 5.20.0
8
- python_version: '3.12'
9
- app_file: app.py
10
- pinned: false
11
- ---
12
-
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Multimodel Setup
3
+ emoji: πŸ¦€
4
+ colorFrom: pink
5
+ colorTo: red
6
+ sdk: gradio
7
+ sdk_version: 5.20.0
8
+ python_version: '3.12'
9
+ app_file: app.py
10
+ pinned: false
11
+ ---
12
+
13
+ # Multi-Model Deepfake Detection & Cross-Generator Evaluation
14
+
15
+ A multi-model deepfake detection framework that combines specialized computer vision architectures to detect AI-generated and manipulated images across diverse generator families (GANs, Diffusion, Midjourney, StyleGAN, ADM).
16
+
17
+ ---
18
+
19
+ ## πŸ’‘ The Core Idea
20
+
21
+ Single deepfake detectors suffer from **generator overfitting**:
22
+ - A model trained on facial GANs (like StyleGAN) often fails on diffusion models (like Midjourney).
23
+ - A model trained on general diffusion artifacts might miss fine-grained facial manipulations.
24
+
25
+ **Our Solution:**
26
+ Instead of relying on a single fallible detector, we run multiple complementary vision models with distinct inductive biases and combine their outputs using **deterministic decision fusion**.
27
+
28
+ ```mermaid
29
+ flowchart TD
30
+ IMG["Input Image"] --> P1["ViT Preprocessor (224x224 Resize)"]
31
+ IMG --> P2["UnivFD Preprocessor (Native CenterCrop + CLIP Norm)"]
32
+ IMG --> P3["ConvNeXt Preprocessor (Native CenterCrop + ImageNet Norm)"]
33
+
34
+ P1 --> M1["ViT Detector<br/>(Facial & GAN Artifacts)"]
35
+ P2 --> M2["UnivFD<br/>(CLIP ViT-L/14 + Linear Probe)"]
36
+ P3 --> M3["ConvNeXt DRCT<br/>(Diffusion Contrastive Base)"]
37
+
38
+ M1 -->|Fake Prob / Vote| FUSION{"Decision Fusion Engine<br/>(src/decision_fusion.py)"}
39
+ M2 -->|Fake Prob / Vote| FUSION
40
+ M3 -->|Fake Prob / Vote| FUSION
41
+
42
+ FUSION --> DECISION["Final Output:<br/>β€’ Prediction: REAL / FAKE<br/>β€’ Fused Confidence Score<br/>β€’ Per-Model Explanation"]
43
+ ```
44
+
45
+ ---
46
+
47
+ ## πŸ” The Base Detectors
48
+
49
+ | Model | Architecture | Specialization / Training | Key Strength |
50
+ |---|---|---|---|
51
+ | **ViT** | Vision Transformer (`dima806`) | Facial deepfakes & GAN faces | High sensitivity on facial boundaries and warping |
52
+ | **UnivFD** | CLIP ViT-L/14 + Linear Classifier | Multi-generator frequency & semantic cues | Universal zero-shot artifact sensitivity |
53
+ | **ConvNeXt DRCT** | ConvNeXt-Base (CVPR 2024) | Diffusion Reconstruction Contrastive Training | Outstanding generalization to diffusion (100% on Midjourney) |
54
+
55
+ ---
56
+
57
+ ## βš™οΈ How It Works: Inference & Fusion Flow
58
+
59
+ ```mermaid
60
+ sequenceDiagram
61
+ autonumber
62
+ actor User
63
+ participant App as Web UI / API (app.py)
64
+ participant Reg as Detector Registry
65
+ participant Models as Base Detectors (ViT, UnivFD, ConvNeXt)
66
+ participant Fusion as Decision Fusion
67
+
68
+ User->>App: Upload Image
69
+ App->>Reg: Request Predictions
70
+ par Run Detectors in Parallel / Safe Sequential
71
+ Reg->>Models: ViT Inference
72
+ Reg->>Models: UnivFD Inference
73
+ Reg->>Models: ConvNeXt DRCT Inference
74
+ end
75
+ Models-->>Reg: Individual Probabilities & Votes
76
+ Reg->>Fusion: Submit {ViT: p1, UnivFD: p2, ConvNeXt: p3}
77
+ Note over Fusion: Apply Strategy (Majority Vote / Mean Prob / Priority)
78
+ Fusion-->>App: Unified Fused Verdict + Confidence Breakdown
79
+ App-->>User: Display Results, Charts & Confidence
80
+ ```
81
+
82
+ ### Fusion Strategies
83
+ 1. **Majority Voting:** Collects discrete votes (`REAL` vs `FAKE`). If a tie occurs, falls back to probability thresholding.
84
+ 2. **Mean Fake Probability:** Computes the unweighted average of calibrated fake probabilities across all active detectors.
85
+ 3. **Domain Priority:** Selects primary detector based on input context (e.g. facial vs non-facial diffusion synthesis).
86
+ 4. **Graceful Degradation:** If any detector fails or checkpoint is absent, the engine falls back gracefully without crashing.
87
+
88
+ ---
89
+
90
+ ## πŸ“Š Evaluation & Parity Verification
91
+
92
+ Our framework was evaluated against both facial GAN benchmarks and external diffusion datasets (Midjourney v5, ADM, ImageNet nature photos):
93
+
94
+ ```mermaid
95
+ flowchart LR
96
+ A["Test Datasets"] --> B["Phase 6: FFHQ vs StyleGAN (Facial)"]
97
+ A --> C["Phase 7: Midjourney + ADM vs ImageNet (Diffusion)"]
98
+
99
+ B --> D["ViT: 99.5% Acc<br/>ConvNeXt: 82.5% Acc"]
100
+ C --> E["ConvNeXt DRCT: 77.0% Acc (100% Midjourney Recall)<br/>UnivFD: 57.5% Acc<br/>ViT: 52.0% Acc"]
101
+ ```
102
+
103
+ - **Reference Parity:** All adapters match official paper implementations (CVPR 2023 UniversalFakeDetect and CVPR 2024 DRCT) with numerical differences $< 10^{-6}$.
104
+ - **Comprehensive Regression Tests:** Automated test suite with 25 unit and contract tests verifying preprocessing shapes, normalization, and fusion edge cases.
105
+
106
+ ---
107
+
108
+ ## πŸš€ Quickstart
109
+
110
+ ### 1. Installation
111
+ ```bash
112
+ git clone https://github.com/Thiru-Selvam-06/Deepfake-Detection.git
113
+ cd Deepfake-Detection
114
+ pip install -r requirements.txt
115
+ ```
116
+
117
+ ### 2. Run the Interactive Web UI
118
+ ```bash
119
+ python app.py
120
+ ```
121
+ Open your browser at `http://localhost:7860`.
122
+
123
+ ### 3. Run Automated Tests
124
+ ```bash
125
+ pytest tests/ -v
126
+ ```
127
+
128
+ ---
129
+
130
+ ## πŸ“ Repository Structure
131
+
132
+ ```text
133
+ β”œβ”€β”€ app.py # Gradio Web Interface
134
+ β”œβ”€β”€ src/
135
+ β”‚ β”œβ”€β”€ convnext_adapter.py # DRCT Native Crop Preprocessing & Inference
136
+ β”‚ β”œβ”€β”€ univfd_adapter.py # UniversalFakeDetect CLIP Preprocessing & Inference
137
+ β”‚ β”œβ”€β”€ vit_adapter.py # Vision Transformer Adapter
138
+ β”‚ β”œβ”€β”€ decision_fusion.py # Multi-Model Voting & Probability Fusion Logic
139
+ β”‚ └── detector_registry.py # Unified Lifecycle & Model Registry
140
+ β”œβ”€β”€ benchmark/ # Labeled test manifests and evaluation results
141
+ β”œβ”€β”€ tests/ # Regression and unit tests for adapters and fusion
142
+ β”œβ”€β”€ docs/ # Detailed scientific evaluation reports
143
+ └── models/ # Pretrained model weights and checkpoints (Git LFS)
144
+ ```