Hugging Face's logo

SpeechAntiSpoofingBenchmarks
/
XLSR-Conformer-TCM-DF

PyTorch
Safetensors
audio
anti-spoofing
deepfake-detection
speech-anti-spoofing-arena
Model card Files
xet
 Community
XLSR-Conformer-TCM-DF / implementation-notes.md
korallll's picture
korallll
Wrapper code (speech-spoof-bench)
6332a34 verified
preview code
|
Raw History Blame Contribute Delete
4.81 kB

XLSR-Conformer-TCM-DF — implementation notes

Plan: speech-spoof-bench/docs/plans/2026-11-01-nine-model-batch-arena-submission.md (model #4; sibling #9 XLSR-Conformer-TCM-LA in ../XLSR-Conformer-TCM-LA/).

Source

  • Code: github.com/ductuantruong/tcm_add @ 726387eb731223a21d551476349f8f8f680a3375 (MIT, LICENSE.source): conformer.py + model.py → _net.py (edits in its header). Diff vs dpda_ga nets/conformer*.py: only (out, attn) returns, the commented-out attn_drop (p = 0 anyway) and the xlsr path → the network is the DPDA-GA-TCM one; _net.py, wrapper, tests, sweep and TRT script are copied from ../DPDA-GA-TCM/.
  • Hyper-parameters = main.py defaults (emb 144, heads 4, kernel 31, 4 encoders); strict load confirms.
  • XLS-R 300M architecture from xlsr2_300m.pt cfg (symlink to ../W2V2-AASIST/, merge_with_parent); base weights not loaded (the checkpoint overwrites all 588 tensors).
  • Weights: the authors' NTU SharePoint folder model_checkpoints/interspeech24_tcm/ (guest link in the tcm_add README; two-step curl with a guest cookie, ../_ckpt_dl/ntu__tcm_add/dl.sh, proxy unset). DF/fixed_length/avg_5_best.pth (1,285,107,735 B, uploaded 2024-07-15) = the authors' own main.py average, re-saved unchanged as model.safetensors (convert.py, weights_only=True, 588 tensors fp32 + int64).

!! DF and LA releases are the same model

  • avg_5_best.pth equals main.py's average of LA/fixed_length/best_{0..4}.pth — i.e. the (2*b1+b2+b3+b4)/5 aliasing formula, see ../XLSR-Conformer-TCM-LA/make_avg.py — to max |Δ| 4.8e-7 (float rounding; num_batches_tracked 14212 = trunc((2*15228+12690+19035+8883)/5) exactly). Plain mean of b0..b4: max |Δ| 0.017. This also confirms the authors really ran the buggy averaging.
  • SharePoint DF/fixed_length/ also holds best_0..4.pth (uploaded 2024-07-21; LA's were uploaded 2024-09-16). Downloaded only to check: they are byte-identical to LA/fixed_length/best_* (_ckpt_dl/ntu__tcm_add/dl_df_best.log).
  • So the LA folder contains the DF-track n-best checkpoints (or the reverse); the authors' distinct LA-track model (RawBoost algo 5) is not in the release. J-SPAW_LA PyTorch scores of the two wrappers: Spearman 1.000000, max |Δ| 6.5e-5. Maintainer decision needed (drop #9, or keep a single entry).

Wrapper decisions

  • Input: first 66,800 samples (source data_utils.Dataset_eval.cut; the paper text says 64,600), shorter clips repeat-tiled (utils.pad). No resampling/normalisation.
  • Score = logits[:, 1] (bona fide = 1, produce_evaluation_file). .to(device) then .eval().
  • cuDNN TF32 disabled in load() (DPDA-GA-TCM finding: TF32 made scores batch-dependent by ~2e-4).
  • Params: 321,213,220 (incl. the frozen 10000×144 positional table; paper's 319.77 M excludes it).
  • 5060 batch-size override uses torch.cuda.get_device_name() (current device).
  • Tests (test_xlsr_conformer_tcm_df.py): pad/crop, strict load, eval mode + param count, cls-token rewrite == source stack, batched == single (1e-4), determinism/finite, sign on 16+16 J-SPAW clips. 7 passed.

Batch sweep (sweep.py, 66,800-sample random input, TF32 off) — see the logs_sweep_g*.txt

card best other
RTX 4070 Ti SUPER (idx 2) 121.0 utt/s @24 118.8 @16, 115.8 @32
RTX 5060 Ti (idx 3) 46.7 utt/s @4 ~31 @ >= 8

Sanity (PyTorch, RTX 4070 Ti SUPER)

  • J-SPAW_LA (full): EER 11.5216% (2397 trials, 0 skipped), ~110 utt/s. (DPDA-GA-TCM 12.90.) Not flipped.
  • InTheWild (full): EER 7.8453% (31779 trials, 0 skipped), ~100 utt/s. No paper ITW number; no authors' score files released, so no per-utterance parity check possible. Paper target 21DF 2.06% not re-run here.

TensorRT (trt_xlsr_conformer_tcm_df.py, sidecar trt_xlsr-conformer-tcm-df.json, TRT 10.16)

Copied from DPDA-GA-TCM's script (config block + per-card ONNX filename so both cards can export concurrently).

card precision best batch utt/s (sweep) build parity Spearman (64 ITW) J-SPAW EER J-SPAW Spearman vs PT / max Δ
RTX 4070 Ti SUPER (idx 2) FP32 (FP16 Spearman 0.99872 < 0.9999) 8 173.3 1.0000 11.5216 (=PT) 0.999998 / 0.026
RTX 5060 Ti (idx 3) FP32 (FP16 Spearman 0.99940 < 0.9999; DPDA-GA-TCM's 5060 FP16 passed at 0.99991, this one does not) 4 121.7 0.99995 11.5216 (=PT) 0.999998 / 0.026
Both cards pass ( ΔEER = 0, Spearman ≥ 0.999). J-SPAW_LA submitted from PyTorch (results_pt/); TRT runs in
results_trt2/, results_trt3/ (cmp_scores.py). Unused FP16 engines deleted. Logs: trt_export_g*.log,
jspaw_trt_g*.log. Helper scripts: run_jspaw.sh <gpu> <mod:Class> <outdir> [dataset], run_sweep.sh <gpu>,
_queue_trt.sh <gpu> <outdir> (all take the shared GPU lock).