| # XLSR-Conformer-TCM-DF — implementation notes |
|
|
| Plan: `speech-spoof-bench/docs/plans/2026-11-01-nine-model-batch-arena-submission.md` (model #4; sibling #9 |
| XLSR-Conformer-TCM-LA in `../XLSR-Conformer-TCM-LA/`). |
|
|
| ## Source |
| - Code: github.com/ductuantruong/tcm_add @ `726387eb731223a21d551476349f8f8f680a3375` (MIT, `LICENSE.source`): |
| `conformer.py` + `model.py` → `_net.py` (edits in its header). Diff vs dpda_ga `nets/conformer*.py`: only |
| (out, attn) returns, the commented-out `attn_drop` (p = 0 anyway) and the xlsr path → the network is the |
| DPDA-GA-TCM one; `_net.py`, wrapper, tests, sweep and TRT script are copied from `../DPDA-GA-TCM/`. |
| - Hyper-parameters = main.py defaults (emb 144, heads 4, kernel 31, 4 encoders); strict load confirms. |
| - XLS-R 300M architecture from `xlsr2_300m.pt` cfg (symlink to `../W2V2-AASIST/`, `merge_with_parent`); base |
| weights not loaded (the checkpoint overwrites all 588 tensors). |
| - Weights: the authors' NTU SharePoint folder `model_checkpoints/interspeech24_tcm/` (guest link in the |
| tcm_add README; two-step curl with a guest cookie, `../_ckpt_dl/ntu__tcm_add/dl.sh`, proxy unset). |
| `DF/fixed_length/avg_5_best.pth` (1,285,107,735 B, uploaded 2024-07-15) = the authors' own main.py average, |
| re-saved unchanged as `model.safetensors` (`convert.py`, `weights_only=True`, 588 tensors fp32 + int64). |
| |
| ## !! DF and LA releases are the same model |
| - `avg_5_best.pth` equals main.py's average of `LA/fixed_length/best_{0..4}.pth` — i.e. the |
| `(2*b1+b2+b3+b4)/5` aliasing formula, see `../XLSR-Conformer-TCM-LA/make_avg.py` — to max |Δ| 4.8e-7 (float |
| rounding; num_batches_tracked 14212 = trunc((2*15228+12690+19035+8883)/5) exactly). Plain mean of b0..b4: |
| max |Δ| 0.017. This also confirms the authors really ran the buggy averaging. |
| - SharePoint `DF/fixed_length/` also holds best_0..4.pth (uploaded 2024-07-21; LA's were uploaded 2024-09-16). |
| Downloaded only to check: they are byte-identical to `LA/fixed_length/best_*` (`_ckpt_dl/ntu__tcm_add/dl_df_best.log`). |
| - So the LA folder contains the DF-track n-best checkpoints (or the reverse); the authors' distinct |
| LA-track model (RawBoost algo 5) is not in the release. J-SPAW_LA PyTorch scores of the two wrappers: |
| Spearman 1.000000, max |Δ| 6.5e-5. Maintainer decision needed (drop #9, or keep a single entry). |
| |
| ## Wrapper decisions |
| - Input: first 66,800 samples (source `data_utils.Dataset_eval.cut`; the paper text says 64,600), shorter |
| clips repeat-tiled (`utils.pad`). No resampling/normalisation. |
| - Score = `logits[:, 1]` (bona fide = 1, `produce_evaluation_file`). `.to(device)` then `.eval()`. |
| - cuDNN TF32 disabled in `load()` (DPDA-GA-TCM finding: TF32 made scores batch-dependent by ~2e-4). |
| - Params: 321,213,220 (incl. the frozen 10000×144 positional table; paper's 319.77 M excludes it). |
| - 5060 batch-size override uses `torch.cuda.get_device_name()` (current device). |
| - Tests (`test_xlsr_conformer_tcm_df.py`): pad/crop, strict load, eval mode + param count, cls-token |
| rewrite == source stack, batched == single (1e-4), determinism/finite, sign on 16+16 J-SPAW clips. 7 passed. |
| |
| ## Batch sweep (`sweep.py`, 66,800-sample random input, TF32 off) — see the `logs_sweep_g*.txt` |
| | card | best | other | |
| |---|---|---| |
| | RTX 4070 Ti SUPER (idx 2) | 121.0 utt/s @24 | 118.8 @16, 115.8 @32 | |
| | RTX 5060 Ti (idx 3) | 46.7 utt/s @4 | ~31 @ >= 8 | |
|
|
| ## Sanity (PyTorch, RTX 4070 Ti SUPER) |
| - J-SPAW_LA (full): EER 11.5216% (2397 trials, 0 skipped), ~110 utt/s. (DPDA-GA-TCM 12.90.) Not flipped. |
| - InTheWild (full): EER 7.8453% (31779 trials, 0 skipped), ~100 utt/s. No paper ITW number; no authors' |
| score files released, so no per-utterance parity check possible. Paper target 21DF 2.06% not re-run here. |
| |
| ## TensorRT (`trt_xlsr_conformer_tcm_df.py`, sidecar `trt_xlsr-conformer-tcm-df.json`, TRT 10.16) |
| Copied from DPDA-GA-TCM's script (config block + per-card ONNX filename so both cards can export concurrently). |
| | card | precision | best batch | utt/s (sweep) | build parity Spearman (64 ITW) | J-SPAW EER | J-SPAW Spearman vs PT / max Δ | |
| |---|---|---|---|---|---|---| |
| | RTX 4070 Ti SUPER (idx 2) | FP32 (FP16 Spearman 0.99872 < 0.9999) | 8 | 173.3 | 1.0000 | 11.5216 (=PT) | 0.999998 / 0.026 | |
| | RTX 5060 Ti (idx 3) | FP32 (FP16 Spearman 0.99940 < 0.9999; DPDA-GA-TCM's 5060 FP16 passed at 0.99991, this one does not) | 4 | 121.7 | 0.99995 | 11.5216 (=PT) | 0.999998 / 0.026 | |
| Both cards pass (|ΔEER| = 0, Spearman ≥ 0.999). J-SPAW_LA submitted from PyTorch (`results_pt/`); TRT runs in |
| `results_trt2/`, `results_trt3/` (`cmp_scores.py`). Unused FP16 engines deleted. Logs: `trt_export_g*.log`, |
| `jspaw_trt_g*.log`. Helper scripts: `run_jspaw.sh <gpu> <mod:Class> <outdir> [dataset]`, `run_sweep.sh <gpu>`, |
| `_queue_trt.sh <gpu> <outdir>` (all take the shared GPU lock). |
|
|
