---
license: mit
pipeline_tag: video-classification
library_name: pytorch
tags:
- deepfake-detection
- audio-visual
- multimodal
- digital-forensics
- safetensors
- experiments
- benchmarks
- fakeavceleb
- cross-dataset
- leave-one-generator-out
---

# DAVID-Net: 135-Run Experimental Registry and Benchmark Archive

Official experimental archive, multi-seed checkpoints, training event streams, and evaluation telemetry for **DAVID-Net** (*Disentangled Audio-Visual Deepfake Network*).

This repository contains the persistent, uncompressed experimental record for the entire **135-run / 5-seed empirical campaign** across 27 experimental configurations (seeds $\mathcal{S} \in \{7, 42, 123, 456, 2024\}$).

* **Source Code Repository:** [GitHub: MIHMahmudEli/david-net-av-v2](https://github.com/MIHMahmudEli/david-net-av-v2)
* **Core Pretrained Model:** [Hugging Face: MIHMahmudEli/david-net-av-v2](https://huggingface.co/MIHMahmudEli/david-net-av-v2)
* **License:** MIT License
* **Format:** SafeTensors (`best_model/model.safetensors` per run), JSON configurations, CSV telemetry

---

## 📌 Experimental Campaign Overview

The empirical validation of DAVID-Net reflects a comprehensive matrix of **135 fully converged training runs** partitioned across four experimental stages:

1. **Stage-0: Quadrant-Aware Contrastive Pretraining (QACP)** (*30 runs*):
   * 6 self-supervised pretraining variants $\times$ 5 random seeds (seeds 7, 42, 123, 456, 2024).
   * Generates proxy synthetic media from pristine recordings using self-blended video and vocoder copy-synthesis speech.
2. **Stage-1: In-Domain Baselines & Proposed Architecture** (*35 runs*):
   * 7 model architectures $\times$ 5 random seeds: Video Probe, Audio Probe, AASIST, EfficientNet-B4, Late Fusion, DAVID-Net Phase A, and DAVID-Net End-to-End Phase B.
   * Evaluated under strict identity-disjoint FakeAVCeleb partitioning (0.0% train/test subject identity overlap).
3. **Systematic Component Ablation Study** (*55 runs*):
   * 11 ablation variants $\times$ 5 random seeds: isolating cross-modal synchronization (`no_sync`), subspace disentanglement (`no_disentangle`), localization heads (`no_loc`), modality dropout (`no_moddrop`), single-task loss (`single_task`), pretraining (`no_qacp`), and sub-component pretraining objectives.
4. **Leave-One-Generator-Out (LOGO) Study** (*15 runs*):
   * 3 withheld generator families (`logo_wav2lip`, `logo_fsgan`, `logo_faceswap`) $	imes$ 5 random seeds to measure zero-shot cross-generator generalization.

---

## 🔬 In-Domain Benchmark Summary ($N=5$ Seeds)

All 21 evaluated model configurations averaged over five independent seeds:

| Model / Configuration | Seeds | Video AUC | Audio AUC | Clip AUC |
| :--- | :---: | :---: | :---: | :---: |
| `aasist` | 5 | — | 1.000 ± 0.000 | 0.748 ± 0.004 |
| `audio-probe` | 5 | — | 0.999 ± 0.001 | 0.732 ± 0.001 |
| `davidnet` | 5 | 0.979 ± 0.003 | 0.999 ± 0.002 | 0.976 ± 0.006 |
| `davidnet-compose_quad` | 5 | 0.979 ± 0.004 | 1.000 ± 0.000 | 0.978 ± 0.003 |
| `davidnet-e2e` | 5 | 0.976 ± 0.005 | 1.000 ± 0.000 | 0.978 ± 0.005 |
| `davidnet-legacy` | 5 | 0.985 ± 0.004 | 1.000 ± 0.000 | 0.985 ± 0.002 |
| `davidnet-logo_faceswap` | 5 | 0.912 ± 0.006 | 1.000 ± 0.000 | 0.903 ± 0.003 |
| `davidnet-logo_fsgan` | 5 | 0.934 ± 0.004 | 1.000 ± 0.000 | 0.932 ± 0.007 |
| `davidnet-logo_wav2lip` | 5 | 0.425 ± 0.018 | 0.994 ± 0.010 | 0.919 ± 0.014 |
| `davidnet-no_disentangle` | 5 | 0.979 ± 0.002 | 1.000 ± 0.000 | 0.979 ± 0.001 |
| `davidnet-no_loc` | 5 | 0.980 ± 0.004 | 1.000 ± 0.000 | 0.981 ± 0.004 |
| `davidnet-no_moddrop` | 5 | 0.980 ± 0.003 | 1.000 ± 0.000 | 0.981 ± 0.004 |
| `davidnet-no_qacp` | 5 | 0.981 ± 0.002 | 1.000 ± 0.000 | 0.983 ± 0.003 |
| `davidnet-no_sync` | 5 | 0.973 ± 0.005 | 1.000 ± 0.000 | 0.975 ± 0.011 |
| `davidnet-qacp_no_copysynth` | 5 | 0.980 ± 0.003 | 0.999 ± 0.001 | 0.979 ± 0.006 |
| `davidnet-qacp_no_mismatch` | 5 | 0.976 ± 0.003 | 1.000 ± 0.000 | 0.977 ± 0.003 |
| `davidnet-qacp_no_selfblend` | 5 | 0.977 ± 0.006 | 1.000 ± 0.001 | 0.978 ± 0.006 |
| `davidnet-single_task` | 5 | 0.978 ± 0.003 | 1.000 ± 0.000 | 0.978 ± 0.005 |
| `effnet-b4` | 5 | 0.996 ± 0.001 | — | 0.926 ± 0.002 |
| `late-fusion` | 5 | 0.914 ± 0.020 | 1.000 ± 0.000 | 0.918 ± 0.015 |
| `video-probe` | 5 | 0.896 ± 0.008 | — | 0.830 ± 0.007 |

### Key Findings
* **Proposed Architecture (DAVID-Net Phase A)**: Achieves **$0.976 \pm 0.006$ Clip AUC**, with **$0.977 \pm 0.003$ Video AUC** and **$0.999 \pm 0.002$ Audio AUC**, outperforming monolithic multimodal Late Fusion ($0.918 \pm 0.015$).
* **Cross-Generator Floor (LOGO)**: Maintains $\ge 90.3\%$ Clip AUC across all withheld generator families under leave-one-generator-out evaluation.
* **Empirical Calibration**: Attains low Expected Calibration Error (ECE, 15 bins) of **$0.032$** (visual) and **$0.024$** (acoustic) on the test split.

---

## 📁 Repository Organization & File Hierarchy

Each of the 135 completed runs is stored in a dedicated directory:

```text
experiments/EXP_{ID}_{NAME}_s{SEED}/
├── best_model/
│   ├── model.safetensors       # Pinned model weights for this seed (SafeTensors)
│   ├── config.json             # Architecture parameters
│   └── meta.json               # Checkpoint metadata & validation performance
├── configs/
│   ├── config.json             # Full experiment hyperparameter configuration
│   └── environment.json        # Execution environment & hardware telemetry
├── metrics/
│   ├── training_history.csv    # Per-epoch loss, learning rates, and validation metrics
│   └── eval_metrics.json       # Final test evaluation metrics
└── figures/                    # Generated ROC, PR, score distribution, and calibration plots
```

### Global Reports & Aggregated Metrics (`reports/`)

* [`reports/raw_results.csv`](reports/raw_results.csv): Master tabular compilation of all 105 evaluated experimental runs.
* [`reports/summary.json`](reports/summary.json): Aggregated statistical metrics, bootstrap confidence intervals, and pairwise DeLong tests ($z$-scores and Holm-corrected $p$-values).
* `reports/tables/`: Machine-readable CSV tables for main results, ablations, external benchmark transfers, LOGO splits, and protocol comparisons.

---

## 💻 Programmatic Usage: Accessing Run Artifacts

You can inspect and load the weights, training logs, or configs of any run programmatically using `huggingface_hub`:

```python
import pandas as pd
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

repo_id = "MIHMahmudEli/davidnet-experiments"

# 1. Download training history for EXP_011 (DAVID-Net Phase A, Seed 42)
history_path = hf_hub_download(
    repo_id=repo_id,
    filename="experiments/EXP_011_davidnet_s42/metrics/training_history.csv"
)
df_history = pd.read_csv(history_path)
print("Training Epochs:", len(df_history))

# 2. Download checkpoint weights (SafeTensors)
weights_path = hf_hub_download(
    repo_id=repo_id,
    filename="experiments/EXP_011_davidnet_s42/best_model/model.safetensors"
)
state_dict = load_file(weights_path)
print(f"Loaded {len(state_dict)} tensor keys.")
```

---

## 📋 Complete 135-Run Campaign Registry

| ID | Configuration | Seed | Status | Test Clip AUC |
|:---|:---|:---:|:---:|:---:|
| EXP_001 | video-probe | 42 | completed | 0.8373 |
| EXP_002 | audio-probe | 42 | completed | 0.7294 |
| EXP_003 | late-fusion | 42 | completed | 0.9307 |
| EXP_004 | aasist | 42 | completed | 0.7471 |
| EXP_005 | effnet-b4 | 42 | completed | 0.9232 |
| EXP_006 | qacp | 42 | completed |  |
| EXP_007 | qacp-no_sync | 42 | completed |  |
| EXP_008 | qacp-no_mismatch | 42 | completed |  |
| EXP_009 | qacp-no_copysynth | 42 | completed |  |
| EXP_010 | qacp-no_selfblend | 42 | completed |  |
| EXP_011 | davidnet | 42 | completed | 0.9725 |
| EXP_012 | davidnet-no_qacp | 42 | completed | 0.9857 |
| EXP_013 | davidnet-no_sync | 42 | completed | 0.9810 |
| EXP_014 | davidnet-no_disentangle | 42 | completed | 0.9802 |
| EXP_015 | davidnet-no_loc | 42 | completed | 0.9799 |
| EXP_016 | davidnet-single_task | 42 | completed | 0.9747 |
| EXP_017 | davidnet-no_moddrop | 42 | completed | 0.9762 |
| EXP_018 | davidnet-compose_quad | 42 | completed | 0.9766 |
| EXP_019 | davidnet-qacp_no_mismatch | 42 | completed | 0.9796 |
| EXP_020 | davidnet-qacp_no_copysynth | 42 | completed | 0.9701 |
| EXP_021 | davidnet-qacp_no_selfblend | 42 | completed | 0.9708 |
| EXP_022 | davidnet-logo_wav2lip | 42 | completed | 0.9274 |
| EXP_023 | davidnet-logo_fsgan | 42 | completed | 0.9350 |
| EXP_024 | davidnet-logo_faceswap | 42 | completed | 0.9012 |
| EXP_025 | qacp-legacy | 42 | completed |  |
| EXP_026 | davidnet-legacy | 42 | completed | 0.9826 |
| EXP_027 | davidnet-e2e | 42 | completed | 0.9793 |
| EXP_028 | video-probe | 123 | completed | 0.8349 |
| EXP_029 | audio-probe | 123 | completed | 0.7336 |
| EXP_030 | late-fusion | 123 | completed | 0.9273 |
| EXP_031 | aasist | 123 | completed | 0.7535 |
| EXP_032 | effnet-b4 | 123 | completed | 0.9259 |
| EXP_033 | qacp | 123 | completed |  |
| EXP_034 | qacp-no_sync | 123 | completed |  |
| EXP_035 | qacp-no_mismatch | 123 | completed |  |
| EXP_036 | qacp-no_copysynth | 123 | completed |  |
| EXP_037 | qacp-no_selfblend | 123 | completed |  |
| EXP_038 | davidnet | 123 | completed | 0.9829 |
| EXP_039 | davidnet-no_qacp | 123 | completed | 0.9797 |
| EXP_040 | davidnet-no_sync | 123 | completed | 0.9578 |
| EXP_041 | davidnet-no_disentangle | 123 | completed | 0.9799 |
| EXP_042 | davidnet-no_loc | 123 | completed | 0.9768 |
| EXP_043 | davidnet-single_task | 123 | completed | 0.9779 |
| EXP_044 | davidnet-no_moddrop | 123 | completed | 0.9803 |
| EXP_045 | davidnet-compose_quad | 123 | completed | 0.9802 |
| EXP_046 | davidnet-qacp_no_mismatch | 123 | completed | 0.9739 |
| EXP_047 | davidnet-qacp_no_copysynth | 123 | completed | 0.9752 |
| EXP_048 | davidnet-qacp_no_selfblend | 123 | completed | 0.9770 |
| EXP_049 | davidnet-logo_wav2lip | 123 | completed | 0.8979 |
| EXP_050 | davidnet-logo_fsgan | 123 | completed | 0.9330 |
| EXP_051 | davidnet-logo_faceswap | 123 | completed | 0.9083 |
| EXP_052 | qacp-legacy | 123 | completed |  |
| EXP_053 | davidnet-legacy | 123 | completed | 0.9865 |
| EXP_054 | davidnet-e2e | 123 | completed | 0.9723 |
| EXP_055 | video-probe | 456 | completed | 0.8204 |
| EXP_056 | audio-probe | 456 | completed | 0.7319 |
| EXP_057 | late-fusion | 456 | completed | 0.9004 |
| EXP_058 | aasist | 456 | completed | 0.7447 |
| EXP_059 | effnet-b4 | 456 | completed | 0.9274 |
| EXP_060 | qacp | 456 | completed |  |
| EXP_061 | qacp-no_sync | 456 | completed |  |
| EXP_062 | qacp-no_mismatch | 456 | completed |  |
| EXP_063 | qacp-no_copysynth | 456 | completed |  |
| EXP_064 | qacp-no_selfblend | 456 | completed |  |
| EXP_065 | davidnet | 456 | completed | 0.9726 |
| EXP_066 | davidnet-no_qacp | 456 | completed | 0.9828 |
| EXP_067 | davidnet-no_sync | 456 | completed | 0.9814 |
| EXP_068 | davidnet-no_disentangle | 456 | completed | 0.9776 |
| EXP_069 | davidnet-no_loc | 456 | completed | 0.9858 |
| EXP_070 | davidnet-single_task | 456 | completed | 0.9819 |
| EXP_071 | davidnet-no_moddrop | 456 | completed | 0.9855 |
| EXP_072 | davidnet-compose_quad | 456 | completed | 0.9828 |
| EXP_073 | davidnet-qacp_no_mismatch | 456 | completed | 0.9802 |
| EXP_074 | davidnet-qacp_no_copysynth | 456 | completed | 0.9833 |
| EXP_075 | davidnet-qacp_no_selfblend | 456 | completed | 0.9840 |
| EXP_076 | davidnet-logo_wav2lip | 456 | completed | 0.9328 |
| EXP_077 | davidnet-logo_fsgan | 456 | completed | 0.9316 |
| EXP_078 | davidnet-logo_faceswap | 456 | completed | 0.9054 |
| EXP_079 | qacp-legacy | 456 | completed |  |
| EXP_080 | davidnet-legacy | 456 | completed | 0.9877 |
| EXP_081 | davidnet-e2e | 456 | completed | 0.9748 |
| EXP_082 | audio-probe | 7 | completed | 0.7315 |
| EXP_083 | qacp-no_selfblend | 7 | completed |  |
| EXP_084 | late-fusion | 2024 | completed | 0.9032 |
| EXP_085 | qacp-no_copysynth | 2024 | completed |  |
| EXP_086 | davidnet-no_qacp | 2024 | completed | 0.9806 |
| EXP_087 | qacp-no_mismatch | 2024 | completed |  |
| EXP_088 | qacp-no_copysynth | 7 | completed |  |
| EXP_089 | qacp | 2024 | completed |  |
| EXP_090 | video-probe | 2024 | completed | 0.8260 |
| EXP_091 | qacp-no_selfblend | 2024 | completed |  |
| EXP_092 | video-probe | 7 | completed | 0.8334 |
| EXP_093 | late-fusion | 7 | completed | 0.9302 |
| EXP_094 | aasist | 2024 | completed | 0.7455 |
| EXP_095 | qacp-no_sync | 2024 | completed |  |
| EXP_096 | qacp-legacy | 2024 | completed |  |
| EXP_097 | davidnet-qacp_no_mismatch | 2024 | completed | 0.9761 |
| EXP_098 | davidnet-logo_wav2lip | 2024 | completed | 0.9096 |
| EXP_099 | davidnet-qacp_no_selfblend | 7 | completed | 0.9742 |
| EXP_100 | qacp | 7 | completed |  |
| EXP_101 | qacp-no_sync | 7 | completed |  |
| EXP_102 | qacp-no_mismatch | 7 | completed |  |
| EXP_103 | qacp-legacy | 7 | completed |  |
| EXP_104 | effnet-b4 | 2024 | completed | 0.9279 |
| EXP_105 | davidnet-legacy | 2024 | completed | 0.9831 |
| EXP_106 | davidnet-compose_quad | 7 | completed | 0.9740 |
| EXP_107 | davidnet-logo_fsgan | 7 | completed | 0.9401 |
| EXP_108 | davidnet-no_moddrop | 7 | completed | 0.9782 |
| EXP_109 | audio-probe | 2024 | completed | 0.7313 |
| EXP_110 | davidnet-logo_wav2lip | 7 | completed | 0.9256 |
| EXP_111 | davidnet-qacp_no_mismatch | 7 | completed | 0.9750 |
| EXP_112 | davidnet-no_loc | 7 | completed | 0.9804 |
| EXP_113 | davidnet-qacp_no_selfblend | 2024 | completed | 0.9837 |
| EXP_114 | davidnet | 2024 | completed | 0.9694 |
| EXP_115 | davidnet-no_disentangle | 2024 | completed | 0.9799 |
| EXP_116 | davidnet-no_moddrop | 2024 | completed | 0.9841 |
| EXP_117 | davidnet-no_sync | 2024 | completed | 0.9826 |
| EXP_118 | aasist | 7 | completed | 0.7507 |
| EXP_119 | davidnet-no_disentangle | 7 | completed | 0.9778 |
| EXP_120 | davidnet-no_sync | 7 | completed | 0.9700 |
| EXP_121 | davidnet-legacy | 7 | completed | 0.9831 |
| EXP_122 | davidnet-qacp_no_copysynth | 7 | completed | 0.9858 |
| EXP_123 | davidnet-compose_quad | 2024 | completed | 0.9770 |
| EXP_124 | davidnet-no_qacp | 7 | completed | 0.9852 |
| EXP_125 | davidnet-qacp_no_copysynth | 2024 | completed | 0.9795 |
| EXP_126 | davidnet-logo_fsgan | 2024 | completed | 0.9220 |
| EXP_127 | davidnet-no_loc | 2024 | completed | 0.9838 |
| EXP_128 | davidnet-single_task | 2024 | completed | 0.9841 |
| EXP_129 | davidnet-e2e | 2024 | completed | 0.9845 |
| EXP_130 | davidnet-logo_faceswap | 7 | completed | 0.9010 |
| EXP_131 | davidnet-single_task | 7 | completed | 0.9730 |
| EXP_132 | effnet-b4 | 7 | completed | 0.9268 |
| EXP_133 | davidnet | 7 | completed | 0.9807 |
| EXP_134 | davidnet-e2e | 7 | completed | 0.9797 |
| EXP_135 | davidnet-logo_faceswap | 2024 | completed | 0.9008 |

---

## ⚖️ Ethics & Responsible AI Statement

All experimental evaluations in this repository were conducted under strict academic research ethics. Seven benchmark corpora were acquired under their respective research end-user licensing agreements; no raw biometric media is redistributed. The system is strictly defensive by design and contains no generative synthesis capability.