Hugging Face's logo

SpeechAntiSpoofingBenchmarks
/
XLSR-Nes2Net-LA

PyTorch
audio
anti-spoofing
deepfake-detection
speech-anti-spoofing-arena
Model card Files
xet
 Community
XLSR-Nes2Net-LA / implementation-notes.md
korallll's picture
korallll
Wrapper code (speech-spoof-bench)
29161c2 verified
preview code
|
Raw History Blame Contribute Delete
4.42 kB

XLSR-Nes2Net-LA — implementation notes

Plan: speech-spoof-bench/docs/plans/2026-11-07-eleven-model-batch-arena-submission.md (model #3).

Source

  • Code: github XuepingZhang/MultiAPI-Spoof @ 1f712e3fff26e237268385b92365a9debd4cb69f, models/XLSR2_Nes2Net_X_ATT.py, config job/xlsr2_nes2net_ATT/few_dkucc.conf (Nes_ratio [8,8], SE_ratio [1], dilation 2, pool_func mean, n_output_logits 2). Code license CC BY-NC 4.0 (README; no LICENSE file → LICENSE.source).
  • Weights: HF XuepingZhang/XLSR-Nes2Net-LA @ ef20423b7a2555f699f217184f3629078bf0e664, epoch_5_1.422.pth (1,289,109,435 B, sha256 c855f654f640b39fa19295445d1c67dc3658060f2282e015208eaedb5a34ddf8). No weight license → not re-hosted; meta.yaml links the pinned HF file. epoch_5_1.422.pth here is a symlink to benchmarks/_ckpt_dl/XuepingZhang__XLSR-Nes2Net-LA/ (the HF repo has no README).
  • Paper arXiv 2512.07352 (BibTeX zhang2025multiapispoofmultiapidataset from the GitHub README).

Port (_net.py, from benchmarks/Nes2Net/_net.py)

  • SSLModel / Bottle2neck / SEModule / ASTP reused from Nes2Net (fairseq cross-version fix, xlsr2_300m.pt symlink to W2V2-AASIST). Added Dropout(0.2) in Bottle2neck (source; eval no-op) and LocalWindowAttention verbatim (W=3, dilation 1, 1 head, D=128, pos_emb 8×128, attn dropout 0.1 = eval no-op, clamped edge indices) between the block stack and the concat.
  • global_attn = nn.MultiheadAttention(1024, 8) is built only because it is in the checkpoint; forward never calls it (tested with a hook).
  • Checkpoint: plain OrderedDict, 998 tensors, all module.-prefixed (DataParallel) → stripped; loaded with torch.load(weights_only=True); 429 ssl_model.model.* tensors. strict=True: 0 missing / 0 unexpected.
  • forward returns (ssl_feat, logits) as the source; score = logits[:, 1] (source label_map bonafide=1, batch_out[:, 1]).
  • .to(device) before .eval() (SSLModel.extract_feat would otherwise flip the SSL to train()).
  • Params: forward path 317,969,191 (meta params_millions); 322,167,591 incl. the unused global_attn (4,198,400); back-end (excl. global_attn) 578,599.

Input

Authors: pad_random — random 64600 crop for long clips (train and eval loaders), repeat-pad short ones, no normalisation. Here: deterministic first 64600 samples, tile-repeat if shorter, empty → zeros. FP32, cuDNN TF32 off.

Source parity (source_parity.py, log source_parity.log)

Authors' Nested_Res2Net_TDNN (from the downloaded source file, strict-loaded from the same checkpoint) vs ours on XLS-R features of 12 In-the-Wild clips: max |Δlogits| = 0 (bit-identical). The front-end is the same fairseq model (only its construction differs).

Batch sweep (sweep.py, 64600 random input, 2026-11-07, logs logs_sweep_g{2,1}.txt)

card best notes
RTX 4070 Ti SUPER (idx 2) 122.8 utt/s @24 plateau 16–64 → batch_size = 24
RTX 5060 Ti (idx 1) 45.9 utt/s @4 ~32 at bs ≥ 8 → 4 on 5060 (set in load)

Sanity (PyTorch)

Full J-SPAW_LA (results_pt/, run on RTX 5060 Ti idx 1): EER 0.376 % (2397 trials, 0 skipped). Copied to results/J-SPAW_LA for submission. Unit tests: test_xlsr_nes2net_la.py 8 passed.

TensorRT (trt_xlsr_nes2net_la.py, sidecar trt_xlsr-nes2net-la.json, TRT 10.16.1.11)

Adapted from Forensics-XLSR-Wild's script (net output tuple → introspected L=2, i=1, n_classes=2). FP16 fails the ranking tolerance (64-clip ITW build parity Spearman 0.9989 / 0.9987) → FP32 on both cards; FP16 engines deleted.

card precision best batch utt/s (sweep) build parity Spearman J-SPAW EER J-SPAW Spearman vs PT / max Δ
RTX 4070 Ti SUPER (idx 2) FP32 32 177.0 0.99995 0.3757 (=PT) 0.999996 / 0.043
RTX 5060 Ti (idx 1) FP32 4 119.8 1.00000 0.3757 (=PT) 0.999992 / 0.109

Scripts: run_sweep.sh, run_pt.sh [gpu], run_trt_export.sh ["2 1"], run_trt_jspaw.sh <gpu>, cmp_scores.py.

meta.yaml

  • trained_on: [ODSS] (training mix TIMIT, ODSS, FoR, AI4T, ASVspoof5 train, MLAAD, MultiAPI-Spoof; only ODSS is an Arena corpus besides ASVspoof5 train, which is not our eval split).
  • In-the-Wild was the dev set for checkpoint selection (file name = ITW EER 1.422 %); maintainer decision: ITW stays ranked, stated in notes, not in trained_on. Self-reported ITW 1.42 %.