Hugging Face's logo

SpeechAntiSpoofingBenchmarks
/
SSL-Mamba

PyTorch
Safetensors
audio
anti-spoofing
deepfake-detection
speech-anti-spoofing-arena
Model card Files
xet
 Community
SSL-Mamba / implementation-notes.md
korallll's picture
korallll
Wrapper code (speech-spoof-bench)
ae9d1a9 verified
preview code
|
Raw History Blame Contribute Delete
4.49 kB

SSL-Mamba — implementation notes (2026-11-01)

Tran, Lolive, Guennec, Sini, Delhay, Marteau, "Leveraging SSL Speech Features and Mamba for Enhanced DeepFake Detection", Interspeech 2025, pp. 5323-5327, doi 10.21437/Interspeech.2025-1703 (DOI goes in paper.arxiv_id, our convention for DOI-only papers). Not XLSR-Mamba (Xiao & Das) — a different model.

Source

  • Code: github.com/hoanmyTran/spoofing_ssl_mamba @ 9afc33de7f74a9b9b3423a9028c10e1d18c1c6b8 (MIT -> LICENSE.source).
  • Weights: HF hoanmyTran/ssl_spoofing_mamba @ 4e72dfcef4f03460e08f68a4a5fbcf3705a3414e interspeech2025.pt (Apache-2.0, 1.27 GB, sha256 f468bddca22696f4e5e83fddb34669c0b400ae90f1ed34d5ef594ab5cac70534), downloaded to benchmarks/_ckpt_dl/hoanmyTran__ssl_spoofing_mamba/. Pickle globals checked first (only OrderedDict, FloatStorage, _rebuild_tensor_v2) then loaded with weights_only=True: flat state_dict, 580 keys. loss_fn.weight ([0.1, 0.9], CrossEntropyLoss buffer) dropped -> 579 tensors -> model.safetensors.
  • Hyper-parameters from model.py / config.yaml (confirmed by checkpoint shapes): 12 Mamba blocks d_model 128, d_state 16, d_conv 4, expand 2 (d_inner 256, dt_rank 8); cgMLP kernels 7/15/23/31, merge 31; MHAP 4 heads, mean+std -> 256; max_len 64600.
  • Parameters: 317,521,282 (no buffers; includes XLS-R masked_spec_embed). params_millions: 317.5213.

Edits (all inference-only; listed in _net.py header)

  1. Lightning/torchmetrics scaffolding removed; plain nn.Module with identical submodule names -> strict load.
  2. XLS-R built from xlsr_300m_config.json (facebook/wav2vec2-xls-r-300m config, saved locally) — no Hub weights; front-end is fully fine-tuned and in the checkpoint. pos_conv uses the new parametrized weight-norm keys (parametrizations.weight.original0/1) -> needs recent transformers (5.3.0 here).
  3. mamba_ssm.Mamba -> _mamba_v1.Mamba (copied byte-identical from XLSR-Mamba-LA/: pure-PyTorch, chunked closed-form selective scan, == selective_scan_ref within 1e-4 on the full net — test). Only the plain Mamba mixer is used here (no Block/RMSNorm): block = Sequential(Mamba, LayerNorm), x = x + LN(Mamba(x)), unidirectional.
  4. multiconv_cgmlp.py, _espnet_shim.py, pooling.py copied from SSL-Feature-Gating/ (same authors; cgMLP identical to this repo's except espnet imports -> shim). pooling.py is not in this repo.
  5. inplace SiLU/SELU -> non-inplace (same values).

Inference conventions

  • Window (maintainer decision 2026-11-01): FIXED first 64,600 samples, repeat-tiled (source dataset.pad). Deviation: the authors' eval (batch 1, no transform) scores the full variable-length utterance. No resampling, no normalisation. FP32. .to(device) then .eval().
  • Score = log_softmax[:,1] - log_softmax[:,0] (source _produce_evaluation_file llr; class 1 = bona fide).

Paper sanity targets (12 blocks, EER %)

19LA 0.11; 21LA fix 2.77 / var 1.78; 21DF fix 1.97 / var 1.51; ITW 5.12 (var).

Results

(2026-11-01; GPUs shared with other agents' jobs, throughputs are lower bounds)

  • Tests (test_ssl_mamba.py): 7/7 pass on both cards (pad, strict load + eval + param count, batched == single 1e-4, chunked == reference scan, determinism, sign/finite).
  • PyTorch sweep (sweep.py): RTX 4070 Ti SUPER ~57 utt/s (plateau bs 4-32) -> batch_size = 16; RTX 5060 Ti ~28.6 utt/s at bs 2-4 -> 4 (set in load()).
  • TensorRT (trt_ssl_mamba.py, sidecar trt_ssl-mamba.json, class trt_ssl_mamba:SSLMambaTRT): FP16 failed the build ranking check (Spearman 0.9995 / 0.9998 < 0.9999) -> FP32 on both cards (fp16 plans deleted). best_batch 32; 70.3 utt/s (4070 Ti S), 59.4 utt/s (5060 Ti). Build parity (64 ITW clips) Spearman 1.0000. TRT warning "Slice ... last_affine/Pad ... incorrect fill value type" = a skipped tactic, harmless. One ONNX kept (ssl-mamba_RTX_4070_Ti_SUPER.onnx; graph is card-independent).
  • Full J-SPAW_LA (2,397 trials): PyTorch EER 5.875% (results_pt/); TRT 4070 Ti S 5.875 (Spearman 0.999999, max |d| 0.022) (results_trt2/); TRT 5060 Ti 5.875 (Spearman 0.999999, max |d| 0.028) (results_trt3/) -> both engines accepted.
  • Full InTheWild (31,779 clips, TRT 4070 Ti S): EER 5.83% (paper 5.12% with full-utterance eval; the 4 s window costs ~0.7 pt). No per-utterance author score files are published -> no per-clip parity check.
  • setup_lane.sh <2|3> drove tests / PT J-SPAW / TRT export / TRT J-SPAW; compare_scores.py = score parity.