Duplicate from hiyanachnani/deepfake-voice-detector-sota

Co-authored-by: Hiya Nachnani <hiyanachnani@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,199 @@
+---
+language: en
+license: apache-2.0
+tags:
+  - audio-classification
+  - deepfake-detection
+  - voice-detection
+  - wav2vec2
+  - transfer-learning
+  - pytorch
+  - speech-processing
+  - binary-classification
+metrics:
+  - accuracy
+  - precision
+  - recall
+  - f1
+  - auc-roc
+datasets:
+  - asvspoof-2021
+  - wavefake-test
+  - audio-deepfake
+  - fake-real-audio
+  - deepfake-audio
+  - combined-real-voices
+  - scenefake
+  - gender-balanced-audio-deepfake
+  - synthetic-speech-commands
+---
+
+# Deepfake Voice Detector - SOTA Transfer Learning
+
+Repository: https://huggingface.co/koyelog/deepfake-voice-detector-sota  
+Model card last updated: 2025-10-31
+
+This repository contains a binary audio classification model trained to distinguish real (bonafide) human voice samples from fake (synthetic / deepfake) voice samples. The model leverages transfer learning from facebook/wav2vec2-base and a lightweight temporal classifier (BiGRU + Multi-Head Attention) on top.
+
+## Model Details
+
+- Model name: Deepfake Voice Detector - SOTA Transfer Learning  
+- HF repo: koyelog/deepfake-voice-detector-sota  
+- Task: Audio Classification (Binary — Real vs Fake)  
+- Base model: facebook/wav2vec2-base (feature extractor)  
+- Architecture:
+  - Wav2Vec2 encoder (facebook/wav2vec2-base) — pretrained speech feature extractor; CNN layers frozen
+  - Bidirectional GRU: 2 layers, 256 hidden units per direction (512 total)
+  - Multi-Head Attention: 8 heads, 512-dimensional embeddings
+  - Classification head:
+    - Linear(512 → 512) + ReLU + BatchNorm + Dropout(0.4)
+    - Linear(512 → 128) + ReLU + BatchNorm + Dropout(0.3)
+    - Linear(128 → 1) + Sigmoid
+- Framework: PyTorch + Transformers
+- Total parameters: ~98.5M
+- Trainable parameters: ~98.5M
+- Input: 4-second audio clip at 16 kHz (single-channel)
+- Output: single probability (0..1) representing likelihood of "fake". Default threshold: 0.5 (0 = Real, 1 = Fake)
+- License: Apache-2.0
+
+## Training Procedure
+
+- Training data: 822,166 audio samples aggregated from 19 datasets (listed below)
+  - Real/Bonafide: 387,422 samples (47.1%)
+  - Fake/Deepfake/Synthetic: 434,744 samples (52.9%)
+- Dataset sources (combined): ASVspoof 2021, WaveFake, Audio-Deepfake, Fake-Real-Audio, Deepfake-Audio, Combined-Real-Voices, Scenefake, Gender-Balanced-Audio-Deepfake, Synthetic-Speech-Commands, and 10+ other Kaggle/academic datasets
+- Data preprocessing:
+  - Resample to 16 kHz
+  - Fixed-length segments: 4 seconds (pad/truncate as required)
+  - Feature extraction: raw audio → wav2vec2 feature frames
+  - Data balancing & augmentation: dataset composition described above; standard augmentations (noise, speed perturbation) used where applicable
+- Train / Val split:
+  - Training: 657,732 samples (80%)
+  - Validation: 164,434 samples (20%)
+- Optimization:
+  - Optimizer: AdamW
+  - Learning rate: 5e-5
+  - Weight decay: 0.01
+  - Batch size: 24
+  - Gradient accumulation: 2 (effective batch size 48)
+  - Epochs: 20
+  - Scheduler: Cosine Annealing with Warm Restarts (T_0=5, T_mult=2)
+  - Loss: Binary Cross-Entropy (BCE)
+  - Mixed precision: supported where hardware permits
+- Hardware: Tesla P100-PCIE-16GB
+- Training time: ~16 hours (single GPU as reported)
+- Random seed and reproducibility: users should set deterministic seeds for fully reproducible runs (not included in this artifact)
+
+## Evaluation Results
+
+The reported validation metrics (expected ranges from final evaluation) are:
+
+- Validation accuracy: 95%–97%
+- Precision: ~0.95
+- Recall: ~0.94
+- F1-score: ~0.94
+- AUC-ROC: ~0.96
+
+Notes:
+- These values represent evaluation on the combined held-out validation split described above. Performance will vary by dataset, language, recording conditions, and unseen manipulation techniques.
+- Reported metrics are aggregated and averaged across the validation partition. Per-dataset metrics (e.g., ASVspoof vs WaveFake) will differ and are not included in this artifact.
+
+## How to Use
+
+Supported input: 4-second audio clip sampled at 16 kHz. Longer or shorter clips should be truncated or padded to 4 seconds before inference.
+
+Example (pseudocode / minimal PyTorch + Transformers usage):
+
+```python
+import torch
+import torchaudio
+from transformers import Wav2Vec2FeatureExtractor
+# model = load your checkpoint wrapped with the BiGRU+Attention classifier
+
+# 1) Load audio and resample to 16k
+waveform, sr = torchaudio.load("example.wav")
+if sr != 16000:
+    waveform = torchaudio.functional.resample(waveform, sr, 16000)
+
+# 2) Ensure 4 seconds length (pad or truncate)
+target_len = 4 * 16000
+if waveform.shape[1] < target_len:
+    pad = target_len - waveform.shape[1]
+    waveform = torch.nn.functional.pad(waveform, (0, pad))
+else:
+    waveform = waveform[:, :target_len]
+
+# 3) Feature extraction (wav2vec2)
+feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("facebook/wav2vec2-base")
+input_values = feature_extractor(waveform.squeeze(0).numpy(), sampling_rate=16000, return_tensors="pt").input_values
+
+# 4) Forward pass through model
+model.eval()
+with torch.no_grad():
+    logits = model(input_values)          # model should return a scalar logit per sample
+    prob = torch.sigmoid(logits).item()   # probability of 'fake' class
+
+prediction = 1 if prob >= 0.5 else 0
+confidence = prob if prediction == 1 else 1 - prob
+```
+
+Model outputs:
+- logits / raw score (float)
+- probability (sigmoid(logit)): float in [0,1]
+- final label: 0 = Real (bonafide), 1 = Fake (deepfake/synthetic)
+- confidence: probability of the predicted class
+
+## Labeling Strategy
+
+- Label 0 — Real / Bonafide:
+  - Human voice recordings from authentic sources (keywords in metadata/filenames: bonafide, real, genuine, human, authentic, original)
+- Label 1 — Fake / Deepfake:
+  - AI-generated, synthetic, or manipulated audio produced by text-to-speech, voice conversion, or other spoofing methods (keywords: spoof, fake, deepfake, synthetic, generated, ai)
+
+Labels were assigned by dataset provider metadata and cross-checked using dataset documentation. Users applying new datasets should ensure consistent labeling and metadata mapping.
+
+## Limitations and Biases
+
+- Clip length sensitivity: The model is optimized for 4-second clips; performance on markedly shorter/longer clips may degrade.
+- Language & accent coverage: Although trained on many datasets and multi-language samples, underrepresented languages/accents in the training corpora can cause degraded performance.
+- Dataset composition: Slight skew towards fake samples (52.9% fake vs 47.1% real) — may increase false positives in certain deployment scenarios.
+- Novel attacks: Not evaluated on zero-shot or post-2025 deepfake generation techniques; performance against new generator families is unknown.
+- Environmental factors: Recording quality, background noise, channel effects, and codecs may affect predictions.
+- Ethical risk: Incorrect or automated use of the model can cause reputational or legal harm. Model outputs should not be used as sole evidence.
+
+## Ethical Considerations
+
+- This model is intended as an assistive tool for verification and detection workflows. Human oversight is essential for any high-stakes decisions.
+- Avoid using the model to definitively accuse individuals of wrongdoing without corroborating evidence.
+- Respect privacy and legal restrictions when processing audio data.
+- Be transparent about limitations, false positive/negative rates, and the potential for demographic biases.
+
+## Hardware & Inference Requirements
+
+- Recommended: GPU with CUDA support for fast inference (e.g., NVIDIA GPUs). CPU inference possible but slower.
+- Approximate memory: ~2 GB GPU VRAM for single-sample inference (depends on implementation).
+- Batch inference recommended for throughput.
+
+## Caveats & Reproducibility
+
+- This model card documents a trained model artifact. If you require retraining or further experiments, use the architecture specification above and provide deterministic seeds and full data provenance to replicate results.
+- Exact training scripts, hyperparameter sweep logs, and raw dataset bundles are not included in this model artifact. Users should exercise caution when assuming identical performance in other contexts.
+
+## Citation
+
+If you use this model, please cite:
+
+@misc{deepfake-voice-detector-sota,
+  author = {koyelog},
+  title = {Deepfake Voice Detector - Transfer Learning with Wav2Vec2},
+  year = {2025},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/koyelog/deepfake-voice-detector-sota}
+}
+
+## Contact
+
+Model owner: koyelog  
+Model hub: https://huggingface.co/koyelog/deepfake-voice-detector-sota
+
+```

config.json

@@ -0,0 +1,13 @@
+{
+  "sample_rate": 16000,
+  "n_mels": 128,
+  "n_fft": 1024,
+  "hop_length": 512,
+  "duration": 4,
+  "batch_size": 64,
+  "num_epochs": 25,
+  "learning_rate": 0.0001,
+  "num_workers": 2,
+  "max_samples_per_dataset": 20000,
+  "weight_decay": 0.01
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25fe1be9d540a4eb4e575e9760446049fca6086491e9466718f15a9954faf501
+size 24550756

training_history.csv

@@ -0,0 +1,26 @@
+epoch,train_loss,train_acc,val_loss,val_acc,overfit_gap,precision,recall,f1,auc
+1,0.4914058956415076,0.7931409062693979,0.25378072014214786,0.9068901303538175,0.11374922408441956,0.7466049382716049,0.9401476875242907,0.8322724926888009,0.9799293558398001
+2,0.17981957488268546,0.9367807859427971,0.10469805875157075,0.9517261137372869,0.01494532779448976,0.9935545476247314,0.8087835211815002,0.8916979110873058,0.989794616800737
+3,0.15118872853546675,0.943859523468462,0.09905789159919794,0.9525855894570978,0.00872606598863579,0.9873269185637175,0.8175281772250291,0.8944403104071436,0.9895628763115077
+4,0.14976802766835354,0.9442653870028171,0.11784015442585437,0.9543522895478203,0.010086902545003107,0.889839970227019,0.9292654488923435,0.9091254752851711,0.988922611482386
+5,0.1445015974348515,0.9462111445351669,0.11057593491178279,0.9499116649954639,0.0037005204602970077,0.9734226947076496,0.8184998056743101,0.8892642246384461,0.9909925116028933
+6,0.1428691015028835,0.9487537602062742,0.11330695173180685,0.95354056247911,0.004786802272835788,0.8583204533745492,0.9712397979012826,0.9112954690491385,0.9913732162588198
+7,0.1347347273028152,0.9528959556892518,0.10731067919603936,0.9623740629327221,0.009478107243470313,0.8877224199288256,0.9694908666925768,0.9268066134125952,0.9931502808379566
+8,0.14318231911582743,0.9528840185264766,0.10890989482584523,0.9668146874850786,0.013930668958601955,0.9207789752316128,0.946366109599689,0.9333972208912314,0.9934382265608983
+9,0.15591263878632355,0.9525736522943227,0.1327253967046556,0.9588884114023779,0.0063147591080552345,0.9164237123420796,0.9162456276719783,0.9163346613545816,0.9895576482037538
+10,0.1579983401289416,0.9564890416845724,0.13107980481099066,0.9631857900014325,0.006696748316860068,0.9899216125419933,0.8589195491643995,0.9197794194152533,0.992137971559635
+11,0.1620871775176709,0.95708589982333,0.11948092563486681,0.9759346798452944,0.01884878002196433,0.9558130400628436,0.9457831325301205,0.9507716350849775,0.9938565797433679
+12,0.1631597144752627,0.9583512390774961,0.12612067496903787,0.9736427445924652,0.015291505514969161,0.9340513983371126,0.9605518849591916,0.9471163058057098,0.9929159878020004
+13,0.15869065325755577,0.9612758439574082,0.1183575700241618,0.9778446258893186,0.01656878193191047,0.9484674329501915,0.9621064904780412,0.9552382789890024,0.995776088731297
+14,0.16028827033236884,0.9620517595377931,0.12368546971460669,0.9748364608699804,0.012784701332187343,0.9261856431075844,0.9753206373882627,0.9501183151916706,0.9961787575905052
+15,0.16127749062317578,0.9620875710261185,0.11676261778466585,0.9792293367712362,0.01714176574511772,0.9564037977136214,0.9591916051301982,0.9577956728436985,0.996268761003168
+16,0.15981666400539138,0.9642840089767464,0.11901161285889585,0.9829059829059829,0.01862197392923648,0.9697802197802198,0.9603575592693354,0.9650458894747118,0.9973943910547347
+17,0.16344723200483846,0.9630186697225803,0.12029080224655024,0.9823807477438762,0.01936207802129586,0.9676914039553554,0.9603575592693354,0.9640105335023895,0.9976485816537309
+18,0.1654804719653268,0.962911235257604,0.12397407956148793,0.9840042018812969,0.021092966623692866,0.971574201137032,0.9630781189273222,0.9673075046355031,0.9978636783081578
+19,0.1666370256995317,0.9630902926992313,0.12440983789813954,0.9828104856037817,0.019720192904550404,0.9582535427039448,0.9724057520404198,0.9652777777777777,0.9977542386360805
+20,0.16776425075540113,0.9640452657212434,0.1238755480409032,0.9832402234636871,0.01919495774244373,0.9805572258969734,0.9506412747765255,0.9653675382338431,0.9979477770344161
+21,0.16945092270194892,0.9634842190708113,0.12649001735376147,0.9824284963949769,0.018944277324165593,0.9622678018575851,0.9663816556548775,0.964320341283692,0.9974480068374301
+22,0.17029010752647136,0.9641885116745452,0.12415509804992414,0.9832402234636871,0.01905171178914189,0.9647218453188603,0.9671589584143023,0.965938864628821,0.9980299813404528
+23,0.1705945418411944,0.9639736427445925,0.1259416123034387,0.9821897531394738,0.018216110394881246,0.9567464114832536,0.9714341235911388,0.9640343264873205,0.9978657510991142
+24,0.17078928051216172,0.9646421238600009,0.127408419281426,0.9824284963949769,0.017786372534975947,0.9566131498470948,0.9726000777302759,0.9645403738677972,0.9979398118584851
+25,0.1705726989416151,0.9645108150694743,0.1269040919340602,0.9839564532301962,0.019445638160721868,0.9664468580294803,0.9683249125534396,0.9673849737914969,0.9979098271228375