Hugging Face's logo

MIHMahmudEli
/
davidnet-experiments

Video Classification
PyTorch
Safetensors
deepfake-detection
audio-visual
multimodal
digital-forensics
experiments
benchmarks
fakeavceleb
cross-dataset
leave-one-generator-out
Model card Files
xet
 Community
MIHMahmudEli commited on
Commit
448ad9a
·
verified ·
1 Parent(s): 9c6e610

README: status

Browse files
Files changed (1) hide show
  1. README.md +147 -160
README.md CHANGED
@@ -1,160 +1,147 @@
1
- ---
2
- license: apache-2.0
3
- library_name: pytorch
4
- tags:
5
- - deepfake-detection
6
- - audio-visual
7
- - multimodal
8
- - contrastive-learning
9
- - temporal-localization
10
- - explainable-ai
11
- - pytorch
12
- pipeline_tag: video-classification
13
- ---
14
-
15
- # DAVID-Net: Deepfake Audio-Visual Invariant Disentanglement Network
16
- ### Official Experiment Archive, Checkpoints, & Verification Repository
17
-
18
- [![Codebase](https://img.shields.io/badge/GitHub-david--net--av--v2-blue?logo=github)](https://github.com/MIHMahmudEli/david-net-av-v2)
19
- [![Live Demo](https://img.shields.io/badge/Vercel-Live_Client-black?logo=vercel)](https://david-net-av.vercel.app/)
20
- [![Inference API](https://img.shields.io/badge/Render-FastAPI_Service-46E3B7?logo=render)](https://davidnet-api.onrender.com/health)
21
- [![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](https://opensource.org/licenses/Apache-2.0)
22
-
23
- This repository serves as the **official, persistent source of truth** for all experimental artifacts, multi-seed checkpoints, evaluation curves, and dataset split manifests for **DAVID-Net** (*Deepfake Audio-Visual Invariant Disentanglement Network*).
24
-
25
- All 81 experimental runs ($N=3$ independent random seeds across 27 distinct ablation and benchmark configurations) are committed here with cryptographic reproducibility.
26
-
27
- ---
28
-
29
- ## 🔬 System Overview
30
-
31
- ```
32
- +-----------------------------+
33
- | Prerecorded Media Stream |
34
- +--------------+--------------+
35
- |
36
- +---------------------+---------------------+
37
- | |
38
- [Video Stream (25 fps)] [Audio Stream (16 kHz)]
39
- | |
40
- [VideoMAE Backbone (Frozen)] [WavLM Backbone (Frozen)]
41
- | |
42
- [Low-Rank Adapters (LoRA)] [Low-Rank Adapters (LoRA)]
43
- | |
44
- +---------------------+---------------------+
45
- |
46
- +-----------------+-----------------+
47
- | DAVID-Net Cross-Attention Core |
48
- | (Disentangled Factorization) |
49
- +-----------------+-----------------+
50
- |
51
- +---------------------------+---------------------------+
52
- | | |
53
- [Video Head (z_v)] [Audio Head (z_a)] [Sync Head (z_c)]
54
- P(Video Fake) P(Audio Fake) Cross-Modal Concordance
55
- | | |
56
- +---------------------------+---------------------------+
57
- |
58
- +-----------------+-----------------+
59
- | 4-Quadrant Attribution Head |
60
- | (RVRA / RVFA / FVRA / FVFA) |
61
- +-----------------+-----------------+
62
- ```
63
-
64
- DAVID-Net addresses cross-generator degradation and modality-inconsistent manipulation through four core innovations:
65
- 1. **Quadrant-Aware Contrastive Pretraining (QACP)**: Pretraining on five generator-agnostic pseudo-classes constructed exclusively from pristine data (RVRA) using neural vocoder copy-synthesis, self-blended video transforms, and cross-clip audio swaps.
66
- 2. **Disentangled Representation Factorization**: Decomposing fused multimodal latent space into three strictly orthogonal subspaces ($z_v \perp z_c$ and $z_a \perp z_c$) isolating video authenticity, audio authenticity, and cross-modal synchronization.
67
- 3. **Multi-Task Attribution & Explainability**: Simultaneous prediction of calibrated per-modality probabilities, 4-quadrant attribution, and temporal manipulation bounding intervals.
68
- 4. **Missing-Modality Robustness**: Learnable null-token projection allowing single-modality execution on audio-only or silent-video inputs without architectural branching.
69
-
70
- ---
71
-
72
- ## 📂 Repository Hierarchy
73
-
74
- ```
75
- MIHMahmudEli/davidnet-experiments/
76
- |-- data/
77
- | |-- split_manifests/ # Subject-disjoint benchmark split manifests
78
- | `-- leakage_audits/ # Rigorous identity and subject leakage verification
79
- |-- experiments/
80
- | |-- EXP_001_late-fusion_s42/ # Per-experiment artifacts
81
- | | |-- best_model/
82
- | | | |-- model.safetensors # Checkpoint weights (BF16/FP32)
83
- | | | `-- config.json # Hyperparameters & seed definitions
84
- | | |-- figures/ # ROC, PR, calibration, and training curves
85
- | | `-- metrics/ # Classification reports and confusion matrices
86
- | |-- ... # 81 complete experiment runs (Seeds 42, 123, 456)
87
- | `-- EXP_081_.../
88
- |-- registry/
89
- | `-- experiments_registry.json # Machine-readable index of all runs & statuses
90
- |-- reports/
91
- | |-- figures/ # Aggregated publication figures (PDF / PNG)
92
- | `-- tables/ # Aggregated benchmark CSV and LaTeX tables
93
- `-- sessions/ # Run session metadata & operational logs
94
- ```
95
-
96
- ---
97
-
98
- ## 📊 Benchmark Summary (N = 3 Independent Seeds)
99
-
100
- ### 1. In-Domain Detection & Attribution (FakeAVCeleb)
101
- * **Clip Detection AUC**: **0.976 +/- 0.006** (EER: 5.8 +/- 0.8%)
102
- * **Video Authenticity AUC**: **0.978 +/- 0.004** (EER: 6.9 +/- 0.6%)
103
- * **Audio Authenticity AUC**: **0.915 +/- 0.012** (EER: 16.1 +/- 1.4%)
104
- * **4-Quadrant Attribution Accuracy**: **82.1 +/- 0.9%** (Macro-F1: 0.814)
105
-
106
- ### 2. Zero-Shot Cross-Dataset Transferability (AUC %)
107
- * **Celeb-DF v2**: 83.6 +/- 1.2%
108
- * **DFDC-10**: 78.4 +/- 0.9%
109
- * **DeepFake-TIMIT (HQ)**: 91.2 +/- 0.8%
110
- * **ASVspoof 2019 LA**: 76.8 +/- 1.5%
111
- * **WaveFake**: 79.3 +/- 1.1%
112
- * **In-the-Wild Real-World**: 74.5 +/- 1.4%
113
-
114
- ---
115
-
116
- ## 🚀 Quickstart: Loading Checkpoints
117
-
118
- ```python
119
- import json
120
- import torch
121
- from huggingface_hub import hf_hub_download
122
- from safetensors.torch import load_file
123
-
124
- REPO_ID = "MIHMahmudEli/davidnet-experiments"
125
- EXP_PATH = "experiments/EXP_011_davidnet_s42/best_model"
126
-
127
- # 1. Download configuration
128
- config_file = hf_hub_download(repo_id=REPO_ID, filename=f"{EXP_PATH}/config.json")
129
- with open(config_file, "r") as f:
130
- config = json.load(f)
131
- print(f"Loaded config for: {config['experiment_name']} (Seed {config['seed']})")
132
-
133
- # 2. Download model weights
134
- weights_file = hf_hub_download(repo_id=REPO_ID, filename=f"{EXP_PATH}/model.safetensors")
135
- state_dict = load_file(weights_file)
136
- print(f"Loaded SafeTensors checkpoint successfully ({len(state_dict)} tensors).")
137
- ```
138
-
139
- ---
140
-
141
- ## 🌐 Connected Ecosystem
142
-
143
- * **Official Source Code**: [GitHub - MIHMahmudEli/david-net-av-v2](https://github.com/MIHMahmudEli/david-net-av-v2)
144
- * **Production Web Application**: [david-net-av.vercel.app](https://david-net-av.vercel.app/)
145
- * **Inference API Microservice**: [davidnet-api.onrender.com](https://davidnet-api.onrender.com/)
146
-
147
- ---
148
-
149
- ## 📜 Citation
150
-
151
- If you use these checkpoints, code, or experimental data in your research, please cite:
152
-
153
- ```bibtex
154
- @article{hasan2026davidnet,
155
- title={Disentangled Audio-Visual Representation Learning for Generalized Deepfake Detection and Multi-Task Forensic Attribution},
156
- author={Hasan, Mohammad Mahmudul and Hossain, Moshin Ibna and Khan, Alif Hasan and Jam, MD. Yeahyea and Shakib, Shadman},
157
- year={2026},
158
- url={https://github.com/MIHMahmudEli/david-net-av-v2}
159
- }
160
- ```
 
1
+ # DAVID-Net-AV - experiment repository (full)
2
+
3
+ Updated 2026-10-06T11:33:30Z by `cpu` session (0.10 h).
4
+
5
+ | status | experiments |
6
+ |---|---|
7
+ | completed | 135 |
8
+
9
+ | ID | name | seed | status | test clip AUC |
10
+ |---|---|---|---|---|
11
+ | EXP_001 | video-probe | 42 | completed | 0.8373 |
12
+ | EXP_002 | audio-probe | 42 | completed | 0.7294 |
13
+ | EXP_003 | late-fusion | 42 | completed | 0.9307 |
14
+ | EXP_004 | aasist | 42 | completed | 0.7471 |
15
+ | EXP_005 | effnet-b4 | 42 | completed | 0.9232 |
16
+ | EXP_006 | qacp | 42 | completed | |
17
+ | EXP_007 | qacp-no_sync | 42 | completed | |
18
+ | EXP_008 | qacp-no_mismatch | 42 | completed | |
19
+ | EXP_009 | qacp-no_copysynth | 42 | completed | |
20
+ | EXP_010 | qacp-no_selfblend | 42 | completed | |
21
+ | EXP_011 | davidnet | 42 | completed | 0.9725 |
22
+ | EXP_012 | davidnet-no_qacp | 42 | completed | 0.9857 |
23
+ | EXP_013 | davidnet-no_sync | 42 | completed | 0.9810 |
24
+ | EXP_014 | davidnet-no_disentangle | 42 | completed | 0.9802 |
25
+ | EXP_015 | davidnet-no_loc | 42 | completed | 0.9799 |
26
+ | EXP_016 | davidnet-single_task | 42 | completed | 0.9747 |
27
+ | EXP_017 | davidnet-no_moddrop | 42 | completed | 0.9762 |
28
+ | EXP_018 | davidnet-compose_quad | 42 | completed | 0.9766 |
29
+ | EXP_019 | davidnet-qacp_no_mismatch | 42 | completed | 0.9796 |
30
+ | EXP_020 | davidnet-qacp_no_copysynth | 42 | completed | 0.9701 |
31
+ | EXP_021 | davidnet-qacp_no_selfblend | 42 | completed | 0.9708 |
32
+ | EXP_022 | davidnet-logo_wav2lip | 42 | completed | 0.9274 |
33
+ | EXP_023 | davidnet-logo_fsgan | 42 | completed | 0.9350 |
34
+ | EXP_024 | davidnet-logo_faceswap | 42 | completed | 0.9012 |
35
+ | EXP_025 | qacp-legacy | 42 | completed | |
36
+ | EXP_026 | davidnet-legacy | 42 | completed | 0.9826 |
37
+ | EXP_027 | davidnet-e2e | 42 | completed | 0.9793 |
38
+ | EXP_028 | video-probe | 123 | completed | 0.8349 |
39
+ | EXP_029 | audio-probe | 123 | completed | 0.7336 |
40
+ | EXP_030 | late-fusion | 123 | completed | 0.9273 |
41
+ | EXP_031 | aasist | 123 | completed | 0.7535 |
42
+ | EXP_032 | effnet-b4 | 123 | completed | 0.9259 |
43
+ | EXP_033 | qacp | 123 | completed | |
44
+ | EXP_034 | qacp-no_sync | 123 | completed | |
45
+ | EXP_035 | qacp-no_mismatch | 123 | completed | |
46
+ | EXP_036 | qacp-no_copysynth | 123 | completed | |
47
+ | EXP_037 | qacp-no_selfblend | 123 | completed | |
48
+ | EXP_038 | davidnet | 123 | completed | 0.9829 |
49
+ | EXP_039 | davidnet-no_qacp | 123 | completed | 0.9797 |
50
+ | EXP_040 | davidnet-no_sync | 123 | completed | 0.9578 |
51
+ | EXP_041 | davidnet-no_disentangle | 123 | completed | 0.9799 |
52
+ | EXP_042 | davidnet-no_loc | 123 | completed | 0.9768 |
53
+ | EXP_043 | davidnet-single_task | 123 | completed | 0.9779 |
54
+ | EXP_044 | davidnet-no_moddrop | 123 | completed | 0.9803 |
55
+ | EXP_045 | davidnet-compose_quad | 123 | completed | 0.9802 |
56
+ | EXP_046 | davidnet-qacp_no_mismatch | 123 | completed | 0.9739 |
57
+ | EXP_047 | davidnet-qacp_no_copysynth | 123 | completed | 0.9752 |
58
+ | EXP_048 | davidnet-qacp_no_selfblend | 123 | completed | 0.9770 |
59
+ | EXP_049 | davidnet-logo_wav2lip | 123 | completed | 0.8979 |
60
+ | EXP_050 | davidnet-logo_fsgan | 123 | completed | 0.9330 |
61
+ | EXP_051 | davidnet-logo_faceswap | 123 | completed | 0.9083 |
62
+ | EXP_052 | qacp-legacy | 123 | completed | |
63
+ | EXP_053 | davidnet-legacy | 123 | completed | 0.9865 |
64
+ | EXP_054 | davidnet-e2e | 123 | completed | 0.9723 |
65
+ | EXP_055 | video-probe | 456 | completed | 0.8204 |
66
+ | EXP_056 | audio-probe | 456 | completed | 0.7319 |
67
+ | EXP_057 | late-fusion | 456 | completed | 0.9004 |
68
+ | EXP_058 | aasist | 456 | completed | 0.7447 |
69
+ | EXP_059 | effnet-b4 | 456 | completed | 0.9274 |
70
+ | EXP_060 | qacp | 456 | completed | |
71
+ | EXP_061 | qacp-no_sync | 456 | completed | |
72
+ | EXP_062 | qacp-no_mismatch | 456 | completed | |
73
+ | EXP_063 | qacp-no_copysynth | 456 | completed | |
74
+ | EXP_064 | qacp-no_selfblend | 456 | completed | |
75
+ | EXP_065 | davidnet | 456 | completed | 0.9726 |
76
+ | EXP_066 | davidnet-no_qacp | 456 | completed | 0.9828 |
77
+ | EXP_067 | davidnet-no_sync | 456 | completed | 0.9814 |
78
+ | EXP_068 | davidnet-no_disentangle | 456 | completed | 0.9776 |
79
+ | EXP_069 | davidnet-no_loc | 456 | completed | 0.9858 |
80
+ | EXP_070 | davidnet-single_task | 456 | completed | 0.9819 |
81
+ | EXP_071 | davidnet-no_moddrop | 456 | completed | 0.9855 |
82
+ | EXP_072 | davidnet-compose_quad | 456 | completed | 0.9828 |
83
+ | EXP_073 | davidnet-qacp_no_mismatch | 456 | completed | 0.9802 |
84
+ | EXP_074 | davidnet-qacp_no_copysynth | 456 | completed | 0.9833 |
85
+ | EXP_075 | davidnet-qacp_no_selfblend | 456 | completed | 0.9840 |
86
+ | EXP_076 | davidnet-logo_wav2lip | 456 | completed | 0.9328 |
87
+ | EXP_077 | davidnet-logo_fsgan | 456 | completed | 0.9316 |
88
+ | EXP_078 | davidnet-logo_faceswap | 456 | completed | 0.9054 |
89
+ | EXP_079 | qacp-legacy | 456 | completed | |
90
+ | EXP_080 | davidnet-legacy | 456 | completed | 0.9877 |
91
+ | EXP_081 | davidnet-e2e | 456 | completed | 0.9748 |
92
+ | EXP_082 | audio-probe | 7 | completed | 0.7315 |
93
+ | EXP_083 | qacp-no_selfblend | 7 | completed | |
94
+ | EXP_084 | late-fusion | 2024 | completed | 0.9032 |
95
+ | EXP_085 | qacp-no_copysynth | 2024 | completed | |
96
+ | EXP_086 | davidnet-no_qacp | 2024 | completed | 0.9806 |
97
+ | EXP_087 | qacp-no_mismatch | 2024 | completed | |
98
+ | EXP_088 | qacp-no_copysynth | 7 | completed | |
99
+ | EXP_089 | qacp | 2024 | completed | |
100
+ | EXP_090 | video-probe | 2024 | completed | 0.8260 |
101
+ | EXP_091 | qacp-no_selfblend | 2024 | completed | |
102
+ | EXP_092 | video-probe | 7 | completed | 0.8334 |
103
+ | EXP_093 | late-fusion | 7 | completed | 0.9302 |
104
+ | EXP_094 | aasist | 2024 | completed | 0.7455 |
105
+ | EXP_095 | qacp-no_sync | 2024 | completed | |
106
+ | EXP_096 | qacp-legacy | 2024 | completed | |
107
+ | EXP_097 | davidnet-qacp_no_mismatch | 2024 | completed | 0.9761 |
108
+ | EXP_098 | davidnet-logo_wav2lip | 2024 | completed | 0.9096 |
109
+ | EXP_099 | davidnet-qacp_no_selfblend | 7 | completed | 0.9742 |
110
+ | EXP_100 | qacp | 7 | completed | |
111
+ | EXP_101 | qacp-no_sync | 7 | completed | |
112
+ | EXP_102 | qacp-no_mismatch | 7 | completed | |
113
+ | EXP_103 | qacp-legacy | 7 | completed | |
114
+ | EXP_104 | effnet-b4 | 2024 | completed | 0.9279 |
115
+ | EXP_105 | davidnet-legacy | 2024 | completed | 0.9831 |
116
+ | EXP_106 | davidnet-compose_quad | 7 | completed | 0.9740 |
117
+ | EXP_107 | davidnet-logo_fsgan | 7 | completed | 0.9401 |
118
+ | EXP_108 | davidnet-no_moddrop | 7 | completed | 0.9782 |
119
+ | EXP_109 | audio-probe | 2024 | completed | 0.7313 |
120
+ | EXP_110 | davidnet-logo_wav2lip | 7 | completed | 0.9256 |
121
+ | EXP_111 | davidnet-qacp_no_mismatch | 7 | completed | 0.9750 |
122
+ | EXP_112 | davidnet-no_loc | 7 | completed | 0.9804 |
123
+ | EXP_113 | davidnet-qacp_no_selfblend | 2024 | completed | 0.9837 |
124
+ | EXP_114 | davidnet | 2024 | completed | 0.9694 |
125
+ | EXP_115 | davidnet-no_disentangle | 2024 | completed | 0.9799 |
126
+ | EXP_116 | davidnet-no_moddrop | 2024 | completed | 0.9841 |
127
+ | EXP_117 | davidnet-no_sync | 2024 | completed | 0.9826 |
128
+ | EXP_118 | aasist | 7 | completed | 0.7507 |
129
+ | EXP_119 | davidnet-no_disentangle | 7 | completed | 0.9778 |
130
+ | EXP_120 | davidnet-no_sync | 7 | completed | 0.9700 |
131
+ | EXP_121 | davidnet-legacy | 7 | completed | 0.9831 |
132
+ | EXP_122 | davidnet-qacp_no_copysynth | 7 | completed | 0.9858 |
133
+ | EXP_123 | davidnet-compose_quad | 2024 | completed | 0.9770 |
134
+ | EXP_124 | davidnet-no_qacp | 7 | completed | 0.9852 |
135
+ | EXP_125 | davidnet-qacp_no_copysynth | 2024 | completed | 0.9795 |
136
+ | EXP_126 | davidnet-logo_fsgan | 2024 | completed | 0.9220 |
137
+ | EXP_127 | davidnet-no_loc | 2024 | completed | 0.9838 |
138
+ | EXP_128 | davidnet-single_task | 2024 | completed | 0.9841 |
139
+ | EXP_129 | davidnet-e2e | 2024 | completed | 0.9845 |
140
+ | EXP_130 | davidnet-logo_faceswap | 7 | completed | 0.9010 |
141
+ | EXP_131 | davidnet-single_task | 7 | completed | 0.9730 |
142
+ | EXP_132 | effnet-b4 | 7 | completed | 0.9268 |
143
+ | EXP_133 | davidnet | 7 | completed | 0.9807 |
144
+ | EXP_134 | davidnet-e2e | 7 | completed | 0.9797 |
145
+ | EXP_135 | davidnet-logo_faceswap | 2024 | completed | 0.9008 |
146
+
147
+ Layout: `data/` frozen manifests + splits (SPLITS_SHA256.json), `registry/experiments.json`, `experiments/EXP_###_<name>_s<seed>/` (`checkpoints/`, `best_model/`, `configs/`, `metrics/`, `predictions/`, `figures/`, `logs/`), `reports/` (manuscript tables + figures).