WST-Graph-Q81 / implementation-notes.md

Wrapper code (speech-spoof-bench)

7e4f282 verified about 3 hours ago

4.71 kB

# WST-Graph-Q81 — implementation notes

Plan: `speech-spoof-bench/docs/plans/2026-11-09-thirteen-model-batch-arena-submission.md` (model #11).

## Source
- HF `kwokho1/wst-graph` @ `080000c1685451c7759f43d2ae7d0f08477cfdf8`, `q81-seed9/wst_graph_q81_seed9_batch32_cuda.ts`
  (sha256 `6cd912bc7a2ecf0fd869ddaa441f50341b5cd07ce764d430f22889cf8dbf22d5`), card `license: mit` (no LICENSE file → `LICENSE.source`). Symlinked here from `_ckpt_dl`.
  Only traced TorchScript programs are released (GitHub `saki-ciallo/WST-Graph` promised); no Python source to vendor.
- Paper: arXiv 2609.29372 (Ng, Song, Feng, Xia, 2026). Trained on ASVspoof 2019 LA (random 4 s crops), 12 epochs,
  focal loss, three seeds (9/114/514). Evaluation windows start at the onset (first 64000 samples).

## Security review of the TorchScript program (before first run)
- Unzipped into a scratch dir and read `code/__torch__/**.py` (the program's `model.code`); also loaded on CPU
  (`torch.jit.load(map_location="cpu")`, no forward) and walked every submodule's `graph`.
- Node kinds: only `aten::*` and `prim::*` (`prim::CallMethod/Constant/GetAttr/ListConstruct/NumToTensor/Tuple*`);
  no `torch.ops` outside aten/prim, no file/OS/network ops, no custom classes beyond the model's own modules
  (`__torch__.models.*`, `__torch__.kymatio.scattering1d.frontend.torch_frontend.ScatteringTorch1D`,
  `__torch__.export_deployment_formats.WSTGraphDeploymentWrapper`, torch.nn modules).
- Pickles (`data.pkl`, `constants.pkl`, `traced_inputs.pkl`, read with pickletools, no execution): globals are only the
  `__torch__` module classes above, `torch.*Storage`, `torch._utils._rebuild_tensor_v2`, `collections.OrderedDict`,
  `torch.jit._pickle.restore_type_tag`. Verdict: clean. `_net.check_graph` re-asserts the aten/prim-only rule on load.
- Traced (not scripted): `torch.device("cuda:0")` constants baked in → CUDA only; traced example input [32, 64000].

## Wrapper (`wst_graph_q81.py:WSTGraphQ81`, `_net.py`)
- Preprocessing = model-card quick start: first 64000 samples, right zero-pad, `input_length` = retained count (int64).
  No normalisation, no resampling. Empty clip (card raises) → zeros with input_length 1.
- Score = `logits[:,0] − logits[:,1]` (class 0 = bona fide). FP32; TF32 off (no effect on this program).
- Batch: the "batch32" file is batch-dynamic (tested B = 1, 4, 7, 16, 32, 48: OK; zero-padding rows with length 1 does
  not change the real rows, |Δ| ≤ 8e-6; companions never change a clip's score). B=1 runs a different cuFFT/GEMM path:
  |Δ| vs B≥2 up to 1.3e-2 on J-SPAW clips, while B≥2 agree to ≤ 4e-5. The wrapper therefore pads a lone clip with one
  zero clip (length 1) and drops its score. Repeat runs are not bitwise identical (cuFFT, ~1e-5).
- "Strict load": all tensors live inside the program; tests assert 166 parameter tensors, **119,867 parameters**
  (paper Table 6 says 119k), aten/prim-only graph, eval mode, file sha256. Buffers (fixed WST filters, grid maps,
  normaliser stats, BN running stats): 9,143,550 elements.
- Seed: the paper reports Table 6 (OOD) as "single-seed results" without naming the seed, other tables as three-seed
  averages → no headline seed → seed 9 (card example), same for Q81 and Q82. Not chosen on Arena data.
## Parity / tests
- `source_parity.py`: the card's quick-start code (verbatim tensor prep, `model(input_values, input_length)`) vs the
  wrapper on 16 J-SPAW_LA clips + a 20000-sample and a 139944-sample clip: max |Δ| vs card per-clip (B=1) 1.30e-2; vs card same batch 0.0
  (the per-clip gap is the B=1 path above). Log: `logs_source_parity.txt`.
- `test_wst_graph_q81.py`: prep crop/pad, sha256, op allow-list + params + eval, batch == single (atol 2e-4), prefix-only,
  determinism (atol 1e-4), bona fide > spoof on ASVspoof2019_LA clips (J-SPAW is near-random for this model). 8 passed.
## Sanity (PyTorch, RTX 5060 Ti idx 3)
- Full J-SPAW_LA (`run_pt.sh`): EER **44.458%** (2397 trials, 0 skipped) — near random, as expected (the paper reports
  ~44–49% on most OOD sets; 19LA 3.07, ITW 46.07). Not inverted: ASVspoof2019_LA 2000-clip eval subset (`subset_eer.py 2000`):
  EER **3.595%** (paper 19LA: 19LA 3.07).
- Training data: ASVspoof 2019 LA train (dev for selection) → no trained_on entries.
## Batch sweep (`sweep.py`, 64000-sample input)
| card | result | choice |
|---|---|---|
| RTX 5060 Ti (idx 3), 2026-11-09, card idle | 150.8 utt/s @32 (plateau 32–128, 3.3 GiB @32); B=1 12.5, B=4 6.5 (first-shape cuFFT plan cost) | `batch_size = 32` |
| RTX 4070 Ti SUPER (idx 2) | not swept (not assigned to this builder; no TRT row) | same 32 |

## TensorRT
Not attempted (plan: no TRT for the TorchScript rows).

	# WST-Graph-Q81 — implementation notes

	Plan: `speech-spoof-bench/docs/plans/2026-11-09-thirteen-model-batch-arena-submission.md` (model #11).

	## Source
	- HF `kwokho1/wst-graph` @ `080000c1685451c7759f43d2ae7d0f08477cfdf8`, `q81-seed9/wst_graph_q81_seed9_batch32_cuda.ts`
	(sha256 `6cd912bc7a2ecf0fd869ddaa441f50341b5cd07ce764d430f22889cf8dbf22d5`), card `license: mit` (no LICENSE file → `LICENSE.source`). Symlinked here from `_ckpt_dl`.
	Only traced TorchScript programs are released (GitHub `saki-ciallo/WST-Graph` promised); no Python source to vendor.
	- Paper: arXiv 2609.29372 (Ng, Song, Feng, Xia, 2026). Trained on ASVspoof 2019 LA (random 4 s crops), 12 epochs,
	focal loss, three seeds (9/114/514). Evaluation windows start at the onset (first 64000 samples).

	## Security review of the TorchScript program (before first run)
	- Unzipped into a scratch dir and read `code/__torch__/**.py` (the program's `model.code`); also loaded on CPU
	(`torch.jit.load(map_location="cpu")`, no forward) and walked every submodule's `graph`.
	- Node kinds: only `aten::` and `prim::` (`prim::CallMethod/Constant/GetAttr/ListConstruct/NumToTensor/Tuple*`);
	no `torch.ops` outside aten/prim, no file/OS/network ops, no custom classes beyond the model's own modules
	(`__torch__.models.*`, `__torch__.kymatio.scattering1d.frontend.torch_frontend.ScatteringTorch1D`,
	`__torch__.export_deployment_formats.WSTGraphDeploymentWrapper`, torch.nn modules).
	- Pickles (`data.pkl`, `constants.pkl`, `traced_inputs.pkl`, read with pickletools, no execution): globals are only the
	`__torch__` module classes above, `torch.*Storage`, `torch._utils._rebuild_tensor_v2`, `collections.OrderedDict`,
	`torch.jit._pickle.restore_type_tag`. Verdict: clean. `_net.check_graph` re-asserts the aten/prim-only rule on load.
	- Traced (not scripted): `torch.device("cuda:0")` constants baked in → CUDA only; traced example input [32, 64000].

	## Wrapper (`wst_graph_q81.py:WSTGraphQ81`, `_net.py`)
	- Preprocessing = model-card quick start: first 64000 samples, right zero-pad, `input_length` = retained count (int64).
	No normalisation, no resampling. Empty clip (card raises) → zeros with input_length 1.
	- Score = `logits[:,0] − logits[:,1]` (class 0 = bona fide). FP32; TF32 off (no effect on this program).
	- Batch: the "batch32" file is batch-dynamic (tested B = 1, 4, 7, 16, 32, 48: OK; zero-padding rows with length 1 does
	not change the real rows, \|Δ\| ≤ 8e-6; companions never change a clip's score). B=1 runs a different cuFFT/GEMM path:
	\|Δ\| vs B≥2 up to 1.3e-2 on J-SPAW clips, while B≥2 agree to ≤ 4e-5. The wrapper therefore pads a lone clip with one
	zero clip (length 1) and drops its score. Repeat runs are not bitwise identical (cuFFT, ~1e-5).
	- "Strict load": all tensors live inside the program; tests assert 166 parameter tensors, 119,867 parameters
	(paper Table 6 says 119k), aten/prim-only graph, eval mode, file sha256. Buffers (fixed WST filters, grid maps,
	normaliser stats, BN running stats): 9,143,550 elements.
	- Seed: the paper reports Table 6 (OOD) as "single-seed results" without naming the seed, other tables as three-seed
	averages → no headline seed → seed 9 (card example), same for Q81 and Q82. Not chosen on Arena data.

	## Parity / tests
	- `source_parity.py`: the card's quick-start code (verbatim tensor prep, `model(input_values, input_length)`) vs the
	wrapper on 16 J-SPAW_LA clips + a 20000-sample and a 139944-sample clip: max \|Δ\| vs card per-clip (B=1) 1.30e-2; vs card same batch 0.0
	(the per-clip gap is the B=1 path above). Log: `logs_source_parity.txt`.
	- `test_wst_graph_q81.py`: prep crop/pad, sha256, op allow-list + params + eval, batch == single (atol 2e-4), prefix-only,
	determinism (atol 1e-4), bona fide > spoof on ASVspoof2019_LA clips (J-SPAW is near-random for this model). 8 passed.

	## Sanity (PyTorch, RTX 5060 Ti idx 3)
	- Full J-SPAW_LA (`run_pt.sh`): EER 44.458% (2397 trials, 0 skipped) — near random, as expected (the paper reports
	~44–49% on most OOD sets; 19LA 3.07, ITW 46.07). Not inverted: ASVspoof2019_LA 2000-clip eval subset (`subset_eer.py 2000`):
	EER 3.595% (paper 19LA: 19LA 3.07).
	- Training data: ASVspoof 2019 LA train (dev for selection) → no trained_on entries.

	## Batch sweep (`sweep.py`, 64000-sample input)
	\| card \| result \| choice \|
	\|---\|---\|---\|
	\| RTX 5060 Ti (idx 3), 2026-11-09, card idle \| 150.8 utt/s @32 (plateau 32–128, 3.3 GiB @32); B=1 12.5, B=4 6.5 (first-shape cuFFT plan cost) \| `batch_size = 32` \|
	\| RTX 4070 Ti SUPER (idx 2) \| not swept (not assigned to this builder; no TRT row) \| same 32 \|

	## TensorRT
	Not attempted (plan: no TRT for the TorchScript rows).