| # SSL-Mamba — implementation notes (2026-11-01) |
|
|
| Tran, Lolive, Guennec, Sini, Delhay, Marteau, "Leveraging SSL Speech Features and Mamba for Enhanced |
| DeepFake Detection", Interspeech 2025, pp. 5323-5327, doi 10.21437/Interspeech.2025-1703 (DOI goes in |
| `paper.arxiv_id`, our convention for DOI-only papers). Not XLSR-Mamba (Xiao & Das) — a different model. |
|
|
| ## Source |
| - Code: github.com/hoanmyTran/spoofing_ssl_mamba @ 9afc33de7f74a9b9b3423a9028c10e1d18c1c6b8 (MIT -> `LICENSE.source`). |
| - Weights: HF hoanmyTran/ssl_spoofing_mamba @ 4e72dfcef4f03460e08f68a4a5fbcf3705a3414e `interspeech2025.pt` |
| (Apache-2.0, 1.27 GB, sha256 f468bddca22696f4e5e83fddb34669c0b400ae90f1ed34d5ef594ab5cac70534), downloaded to |
| `benchmarks/_ckpt_dl/hoanmyTran__ssl_spoofing_mamba/`. Pickle globals checked first (only OrderedDict, |
| FloatStorage, _rebuild_tensor_v2) then loaded with `weights_only=True`: flat state_dict, 580 keys. |
| `loss_fn.weight` ([0.1, 0.9], CrossEntropyLoss buffer) dropped -> 579 tensors -> `model.safetensors`. |
| - Hyper-parameters from model.py / config.yaml (confirmed by checkpoint shapes): 12 Mamba blocks d_model 128, |
| d_state 16, d_conv 4, expand 2 (d_inner 256, dt_rank 8); cgMLP kernels 7/15/23/31, merge 31; MHAP 4 heads, |
| mean+std -> 256; max_len 64600. |
| - Parameters: 317,521,282 (no buffers; includes XLS-R `masked_spec_embed`). `params_millions: 317.5213`. |
|
|
| ## Edits (all inference-only; listed in `_net.py` header) |
| 1. Lightning/torchmetrics scaffolding removed; plain nn.Module with identical submodule names -> strict load. |
| 2. XLS-R built from `xlsr_300m_config.json` (facebook/wav2vec2-xls-r-300m config, saved locally) — no |
| Hub weights; front-end is fully fine-tuned and in the checkpoint. pos_conv uses the new parametrized |
| weight-norm keys (`parametrizations.weight.original0/1`) -> needs recent transformers (5.3.0 here). |
| 3. `mamba_ssm.Mamba` -> `_mamba_v1.Mamba` (copied byte-identical from XLSR-Mamba-LA/: pure-PyTorch, |
| chunked closed-form selective scan, == `selective_scan_ref` within 1e-4 on the full net — test). |
| Only the plain `Mamba` mixer is used here (no Block/RMSNorm): block = Sequential(Mamba, LayerNorm), |
| x = x + LN(Mamba(x)), unidirectional. |
| 4. `multiconv_cgmlp.py`, `_espnet_shim.py`, `pooling.py` copied from SSL-Feature-Gating/ (same authors; |
| cgMLP identical to this repo's except espnet imports -> shim). pooling.py is not in this repo. |
| 5. inplace SiLU/SELU -> non-inplace (same values). |
|
|
| ## Inference conventions |
| - Window (maintainer decision 2026-11-01): FIXED first 64,600 samples, repeat-tiled (source dataset.pad). |
| Deviation: the authors' eval (batch 1, no transform) scores the full variable-length utterance. |
| No resampling, no normalisation. FP32. `.to(device)` then `.eval()`. |
| - Score = log_softmax[:,1] - log_softmax[:,0] (source `_produce_evaluation_file` llr; class 1 = bona fide). |
|
|
| ## Paper sanity targets (12 blocks, EER %) |
| 19LA 0.11; 21LA fix 2.77 / var 1.78; 21DF fix 1.97 / var 1.51; ITW 5.12 (var). |
|
|
| ## Results |
| (2026-11-01; GPUs shared with other agents' jobs, throughputs are lower bounds) |
| - Tests (`test_ssl_mamba.py`): 7/7 pass on both cards (pad, strict load + eval + param count, batched == single |
| 1e-4, chunked == reference scan, determinism, sign/finite). |
| - PyTorch sweep (`sweep.py`): RTX 4070 Ti SUPER ~57 utt/s (plateau bs 4-32) -> `batch_size = 16`; |
| RTX 5060 Ti ~28.6 utt/s at bs 2-4 -> 4 (set in load()). |
| - TensorRT (`trt_ssl_mamba.py`, sidecar `trt_ssl-mamba.json`, class `trt_ssl_mamba:SSLMambaTRT`): FP16 failed the |
| build ranking check (Spearman 0.9995 / 0.9998 < 0.9999) -> **FP32** on both cards (fp16 plans deleted). |
| best_batch 32; 70.3 utt/s (4070 Ti S), 59.4 utt/s (5060 Ti). Build parity (64 ITW clips) Spearman 1.0000. |
| TRT warning "Slice ... last_affine/Pad ... incorrect fill value type" = a skipped tactic, harmless. |
| One ONNX kept (`ssl-mamba_RTX_4070_Ti_SUPER.onnx`; graph is card-independent). |
| - Full J-SPAW_LA (2,397 trials): PyTorch EER 5.875% (`results_pt/`); TRT 4070 Ti S 5.875 (Spearman 0.999999, |
| max |d| 0.022) (`results_trt2/`); TRT 5060 Ti 5.875 (Spearman 0.999999, max |d| 0.028) (`results_trt3/`) |
| -> both engines accepted. |
| - Full InTheWild (31,779 clips, TRT 4070 Ti S): EER 5.83% (paper 5.12% with full-utterance eval; the 4 s |
| window costs ~0.7 pt). No per-utterance author score files are published -> no per-clip parity check. |
| - `setup_lane.sh <2|3>` drove tests / PT J-SPAW / TRT export / TRT J-SPAW; `compare_scores.py` = score parity. |
|
|
