01Yassine's picture
Add files using upload-large-folder tool
b4e8fad verified
|
Raw History Blame Contribute Delete
4.17 kB
---
language:
- en
library_name: pytorch
pipeline_tag: audio-classification
datasets:
- DeepFense/AffectDF
tags:
- deepfense
- speech-deepfake-detection
- wav2vec2
- xls-r
- aasist
- frozen-frontend
---
# AffectDF: frozen XLS-R-300M + AASIST, seed 2
DeepFense binary audio deepfake detector trained separately on the official
**AffectDF train split**, with **validation EER selecting this checkpoint**.
This is one of the seeds 2, 42 and 240; it is not an ensemble.
The frontend is frozen fairseq XLS-R-300M (wav2vec2), with 16 kHz audio,
64,600-sample prefix cropping/repeat padding and no training augmentation.
The backend is AASIST. The saved configuration is the architecture authority.
The MLP experiments use projection [256, 128] and layer normalization; AASIST
uses its own graph-attention backend. Labels are 0=spoof and 1=bonafide.
## Files
- `best_model.pth`: original validation-selected checkpoint, byte-identical to the trained file.
- `config.original.yaml`: original resolved training configuration, retained unchanged.
- `config.yaml`: same architecture with portable relative asset/data/output paths.
- `provenance.json`, `SHA256SUMS`: source/configuration and integrity records.
- `metrics.json`: validation selection value and full ITW/ASV19 test results.
The checkpoint is a DeepFense dictionary including `model_state`, optimizer
state and training metadata. It is not a Transformers `from_pretrained` model.
The entire learned frontend and backend are retained. Intermediate epoch
snapshots are training history, rather than additional model releases.
## Download and use
```bash
hf download DeepFense/AffectDF_Wav2Vec2_AASIST_NoAug_Seed2 --local-dir checkpoints/AffectDF_Wav2Vec2_AASIST_NoAug_Seed2
```
Use [DeepFense](https://github.com/Yaselley/deepfense-framework) master source
at commit `3f2fe1146c48ea4bacc480dd96d806b6c72c09cf`, with its supported
PyTorch environment (the evaluated cluster runtime used PyTorch 2.4.1+cu121).
Before running, edit the portable YAML's `model.frontend.args.ckpt_path` to
your local **original XLS-R-300M initializer**, and set the train/val/test
Parquet paths and output directory for your machine. The initializer is
needed by model construction; this detector's saved frontend weights are
then restored from `best_model.pth`. Do not substitute a small Wav2Vec2 encoder.
From the DeepFense checkout, after those path edits:
```bash
python test.py --config /absolute/path/to/checkpoint/config.yaml \
--checkpoint /absolute/path/to/checkpoint/best_model.pth
```
The framework's current CrossEntropy score is its bonafide logit. The study
also reports a separate logit-difference score, `logit_bonafide-logit_spoof`.
Those two EER definitions must not be mixed. Use strict state loading when
building a custom evaluator; every inference parameter was strictly loaded
for the test results below.
## Results for this seed
Best **validation** EER: 2.9664%.
This is checkpoint selection, not performance on this dataset's test split.
| Full held-out target | Utterances | EER logit difference (%) | EER current-framework score (%) |
|---|---:|---:|---:|
| InTheWild | 31,779 | 21.9545 | 21.9108 |
| ASV19 | 71,237 | 29.3387 | 27.8205 |
The target test partitions were evaluated in full, without retraining,
subsampling or silently dropped audio. EER uses the closest tie-grouped
DET point; higher scores mean more likely bonafide. See `metrics.json` for
protocol hashes, AUC, accuracy, balanced accuracy and macro F1. This card
reports this seed only; it does not pool scores across seeds.
The published ASV19 DeepFense models fine-tune their SSL frontend and use
64,000 samples, unlike these frozen 64,600-sample models. Comparisons are
between the trained recipes, not a controlled frontend-freezing ablation.
## Attribution and use
Training data: [DeepFense/AffectDF](https://huggingface.co/datasets/DeepFense/AffectDF).
That dataset retains its source CC BY-NC 4.0 terms and attribution.
The DeepFense source code has an Apache-2.0 license. This model card does
not infer an additional weights license from the source-code license.
Use is intended for research on audio deepfake detection.