Hugging Face's logo

SpeechAntiSpoofingBenchmarks
/
Forensics-XLSR-Wild

PyTorch
audio
anti-spoofing
deepfake-detection
speech-anti-spoofing-arena
Model card Files
xet
 Community
Forensics-XLSR-Wild / implementation-notes.md
korallll's picture
korallll
Wrapper code (speech-spoof-bench)
f7902a1 verified
preview code
|
Raw History Blame Contribute Delete
4.86 kB

Forensics-XLSR-Wild β€” implementation notes

Plan: speech-spoof-bench/docs/plans/2026-11-01-nine-model-batch-arena-submission.md (model #1).

Source

  • HF eliya/forensics_0.3B_xlsr_wild_deepfake_classifier @ 0cbc87b3fbd683ff8c8c961bcf244a8d5f1bcf02 (the old id eliya/forensics_0.3B_xlsr_wild redirects). CC-BY-NC-4.0 (model-card metadata; no LICENSE file β†’ LICENSE.source records it). Downloaded only checkpoint_epoch_4.safetensors (1.26 GB, sha256 39e37fa5a958c2b46ebb6a5937874c36c39003906dcdd5d0aba6154bc6b2dc21), model.py, inference.py, config.json, README.md into benchmarks/_ckpt_dl/eliya__forensics_0.3B_xlsr_wild_deepfake_classifier/ (the 3.79 GB pickled .pt was not fetched). checkpoint_epoch_4.safetensors here is a hard link to it.
  • model.py β†’ _net.py (edits in its header). Backbone config vendored as xlsr300m_config.json (facebook/wav2vec2-xls-r-300m @ 1a640f32ac3e39899438a2931f9924c02f080a54); no Hub weights loaded.
  • Weight-upload commits are titled "Upload itw_winner_xlsr" (2026-09-01) β†’ checkpoint apparently picked on In-the-Wild; card revisions later replaced the explicit public-dataset list with "proprietary and public".

Wrapper decisions (forensics_xlsr_wild.py:ForensicsXLSRWild)

  • Input = inference.py:load_audio after mono/16 kHz: peak-normalise whole clip wav/(max|wav|+1e-8), then centre crop s=(cur-n)//2 if longer than 80000, repeat-pad (repeat(ceil)[:n]) if shorter. Empty clip β†’ zeros (not in source). No resampling. FP32, cuDNN TF32 off.
  • Score = the single logit (label 1 = real, sigmoid = P(real)); higher = bona fide.
  • Load: checkpoint has 589 tensors; the 4 projection.{0,2}.{weight,bias} (160Γ—160, training-only head, not in model.py β€” the source loads strict=False) are dropped via an asserted allow-list, then strict=True (0 missing / 0 unexpected). The source's strict=False load also reports exactly those 4 as unexpected and no missing keys, so nothing in the released model is lost.
  • Params: 315,926,375 (316.0 M; + 518 BN buffer elements).
  • Source parity (source_parity.py): authors' model.py (from_pretrained base + strict=False) + inference.py:load_audio (torchaudio) vs our wrapper on 12 In-the-Wild clips: max |Ξ”| = 0.0 (bit-identical).

Batch sweep (sweep.py, 80000-sample input, 2026-11-01; logs logs_sweep_g{2,3}_tf{0,1}.txt)

card best notes
RTX 4070 Ti SUPER (idx 2) 97.5 utt/s @24 plateau 8–64; TF32 on/off identical
RTX 5060 Ti (idx 3) 32.7 utt/s @1 ~26 at bs β‰₯ 4 β†’ batch_size 1 on 5060

Sanity (PyTorch, 4070 Ti SUPER)

  • Full J-SPAW_LA: EER 0.188% (2397 trials, 0 skipped), ~82 utt/s end-to-end. Very low β€” the undisclosed training data may include JSUT/JVS-like Japanese speech; not flipped.
  • Full InTheWild (results_pt/InTheWild): EER 1.092% (31,779 trials, 0 skipped) vs self-reported 1.03% β€” reproduces. (InTheWild is n/a* via trained_on.) No per-utterance author score files exist.

TensorRT (trt_forensics_xlsr_wild.py, sidecar trt_forensics-xlsr-wild.json, TRT 10.16.1.11)

Adapted from DPDA-GA-TCM's script; _ExportNet returns (B,1) and n_classes is forced to 1. FP16 fails the ranking tolerance (build Spearman 0.978 on 64 ITW clips β€” the AASIST top-k node selection is precision-sensitive), so both cards use FP32.

card precision best batch utt/s (sweep) build parity Spearman (64 ITW) J-SPAW EER J-SPAW Spearman vs PT / max Ξ”
RTX 4070 Ti SUPER (idx 2) FP32 (FP16 0.978) 8 136.3 0.99991 0.1879 (=PT) 0.999991 / 0.0091
RTX 5060 Ti (idx 3) FP32 (FP16 0.978) 4 91.9 0.99991 0.1879 (=PT) 0.999982 / 0.0323

Both cards pass (|Ξ”EER| = 0 ≀ 0.25, Spearman β‰₯ 0.999). End-to-end J-SPAW runner throughput: PT 4070 ~82, TRT 4070 ~114, TRT 5060 ~64 utt/s (decode-bound). FP16 engines deleted. J-SPAW_LA is submitted from PyTorch scores (results_pt/); TRT runs in results_trt2/, results_trt3/; cmp_scores.py compares. Scripts: run_pt.sh, run_trt_export.sh, run_trt_jspaw.sh <gpu>.

meta.yaml

  • No paper block: the schema makes paper optional but, if present, requires arxiv_id + url + bibtex, and any paper_url/arxiv_id moves a system into the paper-gated tiers. As with Spectra-0, the paperless model is left without paper β†’ Unpublished tier (HF card URL is in code; the card's @misc bibtex is not a paper).
  • trained_on: DFADD, ASVspoof2021_DF, ASVspoof2019_LA, InTheWild (plan Q3). Note: the package's data/submission_meta.schema.json (used by speech-spoof-bench submit's load_meta) does not yet allow trained_on (additionalProperties false); the v5 submission schema does. _submit_one.py also does not copy trained_on (nor drop a null paper) into the submission YAML.