Hugging Face's logo

SpeechAntiSpoofingBenchmarks
/
Forensics-WavLM-OCSoftmax

PyTorch
audio
anti-spoofing
deepfake-detection
speech-anti-spoofing-arena
Model card Files
xet
 Community
Forensics-WavLM-OCSoftmax / implementation-notes.md
korallll's picture
korallll
Wrapper code (speech-spoof-bench)
c512ffb verified
preview code
|
Raw History Blame Contribute Delete
5.12 kB

Forensics-WavLM-OCSoftmax β€” implementation notes

Plan: speech-spoof-bench/docs/plans/2026-11-07-eleven-model-batch-arena-submission.md (model #2). Copy of benchmarks/Forensics-WavLM-Base/ (byte-identical model.py / inference.py); only the checkpoint differs.

Source

  • HF eliya/forensics_0.3B_wavlm_oc_softmax_deepfake_classifier @ 2911b8f1f6112e7095c8ac1061e03b81baff95a2, CC-BY-NC-4.0 (model-card metadata; no LICENSE file β†’ LICENSE.source). Downloaded 2nd_oc_softmax_ce_from16_last.safetensors (sha256 53ed62c92e06f503098c1a4d9f90dad4df6cdfdd6a3db308a2b60adf4f196f9a), model.py, inference.py, config.json, README.md into benchmarks/_ckpt_dl/eliya__forensics_0.3B_wavlm_oc_softmax_deepfake_classifier/; 2nd_oc_softmax_ce_from16_last.safetensors here is a hard link to it (pickled .pt not fetched).
  • model.py is byte-identical to the Base repo's; vs XLSR-Wild only WavLMModel.from_pretrained("microsoft/wavlm-large") and the defaults hidden_dim=320, aasist_gat_dims=(128, 64) differ. inference.py identical to XLSR-Wild's apart from names.
  • model.py β†’ _net.py (edits listed in its header). Backbone config vendored as wavlm_large_config.json (microsoft/wavlm-large @ c1423ed94bb01d80a3f5ce5bc39f6026a0f4828c, config only; no Hub weights loaded).
  • Upload commits are titled "Upload 2nd_oc_softmax". Card: run wavlm_aasist_27_03_oc_softmax_ce_from_16, CE + OC-Softmax, "last" epoch (10) shipped rather than the metric-best (8). First card revision lists SpeechFake, DFADD, CodecFake, MD-CommonVoice, ASVspoof2019-LA; 2026-09-03 SpeechFake, ASVspoof2021, CommonVoice, Aishell; current "proprietary and public".
  • The OC-Softmax centre / loss weights are not shipped; the authors' inference.py scores with the classifier logit, as we do.

Wrapper decisions (forensics_wavlm_ocsoftmax.py:ForensicsWavLMOCSoftmax)

  • Input = inference.py:load_audio after mono/16 kHz: peak-normalise wav/(max|wav|+1e-8), centre crop s=(cur-n)//2 if > 80000, repeat-pad if shorter. Empty clip β†’ zeros (not in source). No resampling. FP32, cuDNN TF32 off.
  • Score = the single logit (label 1 = real, sigmoid = P(real)); higher = bona fide.
  • Load: 655 checkpoint tensors; the 4 projection.{0,2}.{weight,bias} (training-only head, not in model.py) are dropped via an asserted allow-list, then strict=True (0 missing / 0 unexpected; 651 tensors all torch.equal to the checkpoint). The pos_conv weight-norm is stored as parametrizations.weight.original0/1, which is the transformers 5.3 module layout β†’ loaded strictly (test test_pos_conv_weight_norm_loaded), not left at init.
  • Params: 317,257,863 (317.26 M).
  • Source parity (source_parity.py, log logs_source_parity.txt): the authors' model.py with its real WavLMModel.from_pretrained path redirected to a local dir (vendored config + the checkpoint's own wavlm.* tensors), then the full checkpoint loaded strict=False like inference.py (reports exactly the 4 projection.* unexpected, 0 missing), input via inference.py:load_audio (torchaudio) vs our wrapper on 20 In-the-Wild clips (10 bona fide / 10 spoof): max |Ξ”| = 0.0 (bit-identical), 5060 Ti.

Batch sweep (sweep.py, 80000-sample input, 2026-11-07; logs_sweep_g{2,0}.txt)

card best notes
RTX 4070 Ti SUPER (idx 2) 85.0 utt/s @24 plateau 8–64
RTX 5060 Ti (idx 0) 26.3 utt/s @24 flat 4–64 (23.2 @1) β†’ no per-card override

Sanity (PyTorch, 5060 Ti idx 0)

  • Full J-SPAW_LA: EER 0.250% (2397 trials, 0 skipped), ~25.2 utt/s. Not inverted.

TensorRT (trt_forensics_wavlm_ocsoftmax.py, sidecar trt_forensics-wavlm-ocsoftmax.json, TRT 10.16.1.11)

Copied from Base's script (only names changed). FP16 engine outputs NaN (parity Spearman βˆ’0.14, WavLM attention / relative-position bias overflows in half precision) β†’ FP32 on both cards.

card precision best batch utt/s (sweep) build parity Spearman / max Ξ” (64 ITW) J-SPAW EER J-SPAW Spearman vs PT / max Ξ”
RTX 4070 Ti SUPER (idx 2) FP32 (FP16 NaN) 32 123.6 0.99986 / 0.0124 0.250 (=PT) 0.999635 / 0.0373
RTX 5060 Ti (idx 0) FP32 (FP16 NaN) 32 80.1 0.99995 / 0.0030 0.250 (=PT) 0.999873 / 0.0110

Both cards pass (|Ξ”EER| = 0 ≀ 0.25, Spearman β‰₯ 0.999). End-to-end J-SPAW runner throughput: PT 5060 ~25.2, TRT 4070 ~104, TRT 5060 ~72 utt/s. FP16 engines deleted. J-SPAW_LA is submitted from PyTorch scores (results_pt/ β†’ results/); TRT runs in results_trt2/, results_trt0/; cmp_scores.py compares. Scripts: run_pt.sh <gpu>, run_trt_export.sh, run_trt_jspaw.sh <gpu>.

meta.yaml

  • No paper block β†’ Unpublished tier (as XLSR-Wild). checkpoint holds a REHOST_COMMIT placeholder: fill with the commit of the re-hosted SpeechAntiSpoofingBenchmarks/Forensics-WavLM-OCSoftmax repo.
  • trained_on: DFADD, ASVspoof2021_DF, ASVspoof2019_LA, InTheWild (plan: same set as XLSR-Wild). No ITW-selection hint in the upload commits; ITW is kept per the plan (undisclosed mix).