korallll commited on
Commit
c21c77c
Β·
verified Β·
1 Parent(s): 8fa5819

Add SONAR EER badge + result row (Arena)

Browse files
Files changed (1) hide show
  1. README.md +77 -231
README.md CHANGED
@@ -1,116 +1,20 @@
1
  ---
2
- library_name: pytorch
3
- tags:
4
- - audio
5
- - spoofing-detection
6
- - anti-spoofing
7
- - wav2vec2
8
- - aasist
9
- - aasist3
10
  license: apache-2.0
11
- pipeline_tag: audio-classification
12
- model-index:
13
- - name: spectra_aasist3
14
- results:
15
- - task:
16
- type: Speech Antispoofing
17
- dataset:
18
- name: ASVspoof19_LA
19
- type: ASVspoof19_LA
20
- metrics:
21
- - name: Equal Error Rate
22
- type: Equal Error Rate
23
- value: 0.723
24
- - task:
25
- type: Speech Antispoofing
26
- dataset:
27
- name: ASVspoof21_LA
28
- type: ASVspoof21_LA
29
- metrics:
30
- - name: Equal Error Rate
31
- type: Equal Error Rate
32
- value: 4.506
33
- - task:
34
- type: Speech Antispoofing
35
- dataset:
36
- name: ASVspoof21_DF
37
- type: ASVspoof21_DF
38
- metrics:
39
- - name: Equal Error Rate
40
- type: Equal Error Rate
41
- value: 1.998
42
- - task:
43
- type: Speech Antispoofing
44
- dataset:
45
- name: ASVspoof5
46
- type: ASVspoof5
47
- metrics:
48
- - name: Equal Error Rate
49
- type: Equal Error Rate
50
- value: 13.82
51
- - task:
52
- type: Speech Antispoofing
53
- dataset:
54
- name: ADD2022
55
- type: ADD2022
56
- metrics:
57
- - name: Equal Error Rate
58
- type: Equal Error Rate
59
- value: 15.187
60
- - task:
61
- type: Speech Antispoofing
62
- dataset:
63
- name: In-the-Wild
64
- type: In-the-Wild
65
- metrics:
66
- - name: Equal Error Rate
67
- type: Equal Error Rate
68
- value: 0.961
69
- - task:
70
- type: Speech Antispoofing
71
- dataset:
72
- name: AD2R1
73
- type: AD2R1
74
- metrics:
75
- - name: Equal Error Rate
76
- type: Equal Error Rate
77
- value: 0.727
78
- - task:
79
- type: Speech Antispoofing
80
- dataset:
81
- name: AD2R2
82
- type: AD2R2
83
- metrics:
84
- - name: Equal Error Rate
85
- type: Equal Error Rate
86
- value: 1.806
87
- - task:
88
- type: Speech Antispoofing
89
- dataset:
90
- name: AD3R1
91
- type: AD3R1
92
- metrics:
93
- - name: Equal Error Rate
94
- type: Equal Error Rate
95
- value: 6.427
96
- - task:
97
- type: Speech Antispoofing
98
- dataset:
99
- name: AD3R2
100
- type: AD3R2
101
- metrics:
102
- - name: Equal Error Rate
103
- type: Equal Error Rate
104
- value: 12.968
105
  ---
106
 
107
- ## πŸ›‘οΈ Speech Anti-Spoofing Arena
108
-
109
- Independently evaluated on the reproducible [Speech Anti-Spoofing Arena](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3) (scores
110
- sha-pinned; reproduced via `speech-spoof-bench reproduce --scoring`). As a pre-release /
111
- unpublished model it is listed in the **πŸ”“ Unpublished / Proprietary** tier (unranked).
112
 
113
  [![EER% 0.00 on CD-ADD](https://img.shields.io/badge/EER%25%20on%20CD--ADD-0.00%25-brightgreen)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
 
114
  [![EER% 0.97 on ASVspoof2019_LA](https://img.shields.io/badge/EER%25%20on%20ASVspoof2019__LA-0.97%25-brightgreen)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
115
  [![EER% 1.20 on InTheWild](https://img.shields.io/badge/EER%25%20on%20InTheWild-1.20%25-brightgreen)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
116
  [![EER% 4.30 on ASVspoof2021_DF](https://img.shields.io/badge/EER%25%20on%20ASVspoof2021__DF-4.30%25-green)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
@@ -118,134 +22,76 @@ unpublished model it is listed in the **πŸ”“ Unpublished / Proprietary** tier (u
118
  [![arena tier](https://img.shields.io/endpoint?url=https://speechantispoofingbenchmarks-speechantispoofingarena.hf.space/badge/spectra-aasist3/tier.json)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
119
  [![arena rank](https://img.shields.io/endpoint?url=https://speechantispoofingbenchmarks-speechantispoofingarena.hf.space/badge/spectra-aasist3/rank.json)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
120
 
121
- | Dataset | EER % | Trials |
122
- |---|---|---|
123
- | CD-ADD | 0.00 | 20,786 |
124
- | ASVspoof2019_LA | 0.97 | 71,237 |
125
- | InTheWild | 1.20 | 31,779 |
126
- | ASVspoof2021_DF | 4.30 | 611,829 |
127
- | ASVspoof2021_LA | 4.38 | 181,566 |
128
-
129
- ## Model Card: Spectra-AASIST3 (anti-spoofing / bonafide vs spoof)
130
-
131
- `Spectra-AASIST3` is a model for **speech spoofing detection** (binary classification: `bonafide` vs `spoof`) from **raw audio waveforms**. Architecture: SSL encoder (`Wav2Vec2`) β†’ MLP projection β†’ `AASIST3` 2-class classifier.
132
-
133
- - **Input**: waveform \(float32\), shape `(batch, num_samples)` (typically 16 kHz).
134
- - **Output**: logits of shape `(batch, 2)`, where **index 0 = spoof**, **index 1 = bonafide**.
135
-
136
- On first run, the model will automatically download the SSL encoder `facebook/wav2vec2-xls-r-300m` via `transformers`.
137
-
138
- ## Evaluation Results
139
-
140
- | Model | ASVspoof19 LA | ASVspoof21 LA | ASVspoof21 DF | ASVspoof5 | ADD2022 | In-the-Wild | AD2R1 | AD2R2 | AD3R1 | AD2R2 |
141
- |-----------|--------|--------|--------|--------|--------|--------|--------|--------|--------|--------|
142
- | [Res2TCNGuard](https://github.com/mtuciru/Res2TCNGuard) | 7.487 | 19.130 | 19.883 | 37.620 | 49.538 | 49.246 | 34.683 | 35.343 | 48.051 | 39.558 |
143
- | [AASIST3](https://huggingface.co/lab260/AASIST3) | 27.585 | 37.407 | 33.099 | 41.001 | 47.192 | 39.626 | 36.581 | 37.351 | 41.333 | 44.278 |
144
- | [XSLS](https://github.com/QiShanZhang/SLSforASVspoof-2021-DF) | 0.231 | 7.714 | 4.220 | 17.688 | 33.951 | 7.453 | 14.386 | 15.743 | 19.368 | 21.095 |
145
- | [TCM-ADD](https://github.com/ductuantruong/tcm_add) | **0.152** | 6.655 | 3.444 | 19.505 | 35.252 | 7.767 | 16.951 | 17.688 | 21.913 | 18.627 |
146
- | [DF Arena 1B](https://huggingface.co/Speech-Arena-2025/DF_Arena_1B_V_1) | 43.793 | 40.137 | 42.994 | 35.333 | 42.139 | 17.598 | 12.442 | 13.292 | 33.381 | 43.42 |
147
- | [Spectra-0](https://huggingface.co/lab260/spectra_0) | 0.181 | 6.475 | 5.410 | 14.426 | **14.716** | 1.026 | 1.578 | 2.372 | 6.535 | 15.154 |
148
- | **[Spectra-AASIST](https://huggingface.co/lab260/Spectra-AASIST)** | 0.159 | 5.164 | 2.568 | 14.056 | 15.205 | 1.461 | 0.939 | **1.802** | **6.427** | **12.968** |
149
- | **Spectra-AASIST3** | 0.723 | **4.506** | **1.998** | **13.82** | 15.187 | **0.961** | **0.727** | 1.806 | 6.502 | 14.481 |
150
-
151
- ## Quickstart
152
-
153
- ### Clone from Hugging Face
154
-
155
- This repository is hosted on Hugging Face Hub: `https://huggingface.co/lab260/spectra_aasist3`.
156
-
157
- ```bash
158
- git lfs install
159
- git clone https://huggingface.co/lab260/spectra_aasist3
160
- cd spectra_aasist3
161
- ```
162
-
163
- ### Install dependencies
164
-
165
- ```bash
166
- pip install -U torch torchaudio transformers huggingface_hub safetensors soundfile
167
- ```
168
-
169
- ### Single-file inference (example preprocessing)
 
 
 
 
 
170
 
171
  ```python
172
- import random
173
- import torch
174
- import torchaudio
175
- import soundfile as sf
176
-
177
- from model import spectra_aasist3
178
-
179
-
180
- def pad_random(x: torch.Tensor, max_len: int = 64600) -> torch.Tensor:
181
- # x: (num_samples,) or (1, num_samples)
182
- if x.ndim > 1:
183
- x = x.squeeze()
184
- x_len = x.shape[0]
185
- if x_len >= max_len:
186
- start = random.randint(0, x_len - max_len)
187
- return x[start:start + max_len]
188
- num_repeats = int(max_len / x_len) + 1
189
- return x.repeat(num_repeats)[:max_len]
190
-
191
-
192
- def load_audio_mono(path: str) -> torch.Tensor:
193
- audio, sr = sf.read(path, dtype="float32")
194
- audio = torch.from_numpy(audio)
195
- if audio.ndim > 1:
196
- # (num_samples, channels) -> mono
197
- audio = audio.mean(dim=1)
198
- if sr != 16000:
199
- audio = torchaudio.functional.resample(audio, sr, 16000)
200
- return audio
201
-
202
-
203
- device = "cuda" if torch.cuda.is_available() else "cpu"
204
- model = spectra_aasist3.from_pretrained(pretrained_model_name_or_path=".").eval().to(device)
205
-
206
- audio = load_audio_mono("path/to/audio.wav")
207
- audio = torchaudio.functional.preemphasis(audio.unsqueeze(0)) # (1, T)
208
- audio = pad_random(audio.squeeze(0), 64600).unsqueeze(0) # (1, 64600)
209
-
210
- with torch.inference_mode():
211
- logits = model(audio.to(device)) # (1, 2)
212
- score_spoof = logits[0, 0].item()
213
- score_bonafide = logits[0, 1].item()
214
-
215
- print({"score_bonafide": score_bonafide, "score_spoof": score_spoof})
216
  ```
217
 
218
- ## Threshold-based classification (and how to tune it)
219
-
220
- In `model.py`, the `SpectraAASIST3` class provides `classify()` with a **default threshold** chosen as an β€œoptimal” value for the original setting:
221
-
222
- - **Default threshold**: `-1.460938` (it thresholds `logit_bonafide = logits[:, 1]`)
223
- - **Note**: this threshold **may not be optimal** on a different dataset/domain. It’s recommended to tune the threshold on your dataset using **EER** (Equal Error Rate) or a target FAR/FRR.
224
-
225
- Example:
226
-
227
- ```python
228
- with torch.inference_mode():
229
- pred = model.classify(audio.to(device), threshold=-1.460938) # 1=bonafide, 0=spoof
230
- ```
231
-
232
- ### Tuning the threshold via EER (typical workflow)
233
-
234
- 1) Run the model on a labeled set and collect scores for both classes.
235
-
236
- 2) Compute EER and the threshold
237
-
238
- ## Limitations and notes
239
-
240
- - This is a **pre-release** model.
241
- - Significantly stronger models are planned for **Q3–Q4 2026** β€” stay tuned.
242
 
243
  ## License
244
 
245
- MIT (see the `license` field in the model repo header).
246
-
247
- ## Contacts
248
-
249
- TG channel: https://t.me/korallll_ai
250
- email: k.n.borodin@mtuci.ru
251
- website: https://lab260.ru/
 
1
  ---
 
 
 
 
 
 
 
 
2
  license: apache-2.0
3
+ tags:
4
+ - audio
5
+ - anti-spoofing
6
+ - audio-deepfake-detection
7
+ - speech
8
+ - asvspoof
9
+ - wav2vec2
10
+ - aasist
11
+ - kan
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ---
13
 
14
+ # Spectra-AASIST3
 
 
 
 
15
 
16
  [![EER% 0.00 on CD-ADD](https://img.shields.io/badge/EER%25%20on%20CD--ADD-0.00%25-brightgreen)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
17
+ [![EER% 0.44 on SONAR](https://img.shields.io/badge/EER%25%20on%20SONAR-0.44%25-brightgreen)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
18
  [![EER% 0.97 on ASVspoof2019_LA](https://img.shields.io/badge/EER%25%20on%20ASVspoof2019__LA-0.97%25-brightgreen)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
19
  [![EER% 1.20 on InTheWild](https://img.shields.io/badge/EER%25%20on%20InTheWild-1.20%25-brightgreen)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
20
  [![EER% 4.30 on ASVspoof2021_DF](https://img.shields.io/badge/EER%25%20on%20ASVspoof2021__DF-4.30%25-green)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
 
22
  [![arena tier](https://img.shields.io/endpoint?url=https://speechantispoofingbenchmarks-speechantispoofingarena.hf.space/badge/spectra-aasist3/tier.json)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
23
  [![arena rank](https://img.shields.io/endpoint?url=https://speechantispoofingbenchmarks-speechantispoofingarena.hf.space/badge/spectra-aasist3/rank.json)](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3)
24
 
25
+ **Spectra-AASIST3** β€” a speech anti-spoofing model pairing a **wav2vec 2.0 (XLS-R-300m)**
26
+ self-supervised front-end with a **KAN-enhanced AASIST** (KAN-AASIST) back-end. The model
27
+ takes a raw speech waveform and returns a score where **higher = more bona fide**.
28
+
29
+ - **Code / checkpoint:** [`lab260/Spectra-AASIST3`](https://huggingface.co/lab260/Spectra-AASIST3)
30
+ (`model.safetensors`, self-contained β€” bundles the SSL encoder weights)
31
+ - **Paper:** none β€” **pre-release / unpublished** model, so it appears in the Arena's
32
+ πŸ”“ *Unpublished / Proprietary* tier (listed but **unranked**, regardless of score).
33
+ - **Parameters:** ~318.95 M
34
+
35
+ The exact wrapper that produced the Arena scores is in
36
+ [`spectra_aasist3.py`](./spectra_aasist3.py); the vendored network is
37
+ [`spectra_aasist3_net.py`](./spectra_aasist3_net.py) (copied from the source `model.py`).
38
+
39
+ ## Architecture
40
+
41
+ 1. **wav2vec 2.0 XLS-R-300m front-end** β€” HF `transformers` `Wav2Vec2Model`
42
+ (`facebook/wav2vec2-xls-r-300m`), producing 1024-d frame features (the base arch is
43
+ fetched at init, then every weight is overwritten by the checkpoint).
44
+ 2. **MLP bridge** β€” a single-layer `Linear(1024 β†’ 128)` projection (SELU, dropout 0.1).
45
+ 3. **KAN-AASIST back-end** β€” max-pool, a RawNet2-style residual encoder, spectral (GAT-S)
46
+ and temporal (GAT-T) graph-attention layers with graph pooling, four parallel
47
+ inference branches with learnable master tokens, and a Kolmogorov-Arnold (KAN)
48
+ output layer.
49
+ 4. The 2-logit output is read at **index 1 = bona fide**.
50
+
51
+ ## How scores are produced
52
+
53
+ - **Input:** raw audio at 16 kHz mono. **Preemphasis (0.97)** is applied to the full
54
+ waveform (matching the source README eval pipeline), then a **deterministic
55
+ first-64,600-sample window** (~4.04 s; tile-repeat if shorter β€” no random crop).
56
+ - **No resampling** in the wrapper (audio arrives at `expected_sample_rate = 16000`).
57
+ - **Output:** 2-class logits; the bona-fide logit (index 1) is the score.
58
+ - `batch_size = 24` (throughput plateaus ~50 utt/s for bs β‰₯ 16 on an RTX 4070 Ti SUPER).
59
+
60
+ ## Benchmark result (Speech Anti-Spoofing Arena)
61
+
62
+ Evaluated through the reproducible
63
+ [Speech Anti-Spoofing Arena](https://huggingface.co/spaces/SpeechAntiSpoofingBenchmarks/SpeechAntiSpoofingArena?system=spectra-aasist3).
64
+ Each result is sha-pinned and reproducible from the score file via
65
+ `speech-spoof-bench reproduce --scoring`.
66
+
67
+ | Dataset | Split | EER % | Trials | Skipped | Notes |
68
+ |---|---|---|---|---|---|
69
+ | CD-ADD | test | **0.00** | 20,786 | 0 | modern neural-TTS deepfake |
70
+ | SONAR | test | **0.44** | 3,948 | 0 | multilingual real-world deepfake |
71
+ | ASVspoof2019_LA | test | **0.97** | 71,237 | 0 | in-domain family |
72
+ | InTheWild | test | **1.20** | 31,779 | 0 | out-of-domain (real-world) |
73
+ | ASVspoof2021_DF | test | **4.30** | 611,829 | 0 | cross-dataset (deepfake) |
74
+ | ASVspoof2021_LA | test | **4.38** | 181,566 | 0 | cross-dataset (logical access) |
75
+
76
+ ## Usage
77
+
78
+ The wrapper loads weights from the Hub via `PyTorchModelHubMixin`:
79
 
80
  ```python
81
+ import numpy as np
82
+ from spectra_aasist3 import SpectraAASIST3 # spectra_aasist3.py + spectra_aasist3_net.py
83
+
84
+ m = SpectraAASIST3()
85
+ m.load() # from_pretrained("lab260/Spectra-AASIST3")
86
+ audio = np.random.randn(48000).astype(np.float32) # float32 mono 16 kHz
87
+ print(m.score_batch([audio], [16000])[0]) # higher = more bona fide
88
+ m.unload()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
89
  ```
90
 
91
+ Internally the wrapper applies preemphasis, windows to 64,600 samples, runs the
92
+ network, and returns `logits[:, 1]` (class 1 = bona fide). [`spectra_aasist3.py`](./spectra_aasist3.py)
93
+ is the exact `speech_spoof_bench` model that produced the Arena `scores.txt`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
94
 
95
  ## License
96
 
97
+ Apache-2.0 β€” see the [source repository](https://huggingface.co/lab260/Spectra-AASIST3).