AADD v3: Whisper-small frame-level degradation detector (12 classes, 20 ms)

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+demo.gif filter=lfs diff=lfs merge=lfs -text
+report_light.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,134 @@
+---
+license: mit
+language:
+  - en
+  - ru
+library_name: pytorch
+pipeline_tag: audio-classification
+base_model: openai/whisper-small
+tags:
+  - audio
+  - speech
+  - audio-quality
+  - degradation-detection
+  - data-curation
+  - sound-event-detection
+  - whisper
+datasets:
+  - openslr/librispeech_asr
+  - istupakov/russian_librispeech
+metrics:
+  - average_precision
+  - roc_auc
+---
+
+# AADD v3: which audio degradation, and where
+
+**AADD** (Automatic Audio Degradation Detection) finds **12 kinds of degradation in
+speech recordings every 20 ms**: background noise, music, reverberation, clipping,
+low-pass / narrow band, high-pass, codec artefacts, packet loss, mains hum, clicks,
+pitch shift and time stretch. Several can be present at once. It is meant for
+cleaning speech datasets (ASR, TTS) and for auditing recordings.
+
+<p align="center"><img src="demo.gif" width="860" alt="AADD v3 localising noise, reverberation, clipping, codec artefacts and clicks in a 14-second clip"></p>
+
+* Code, CLI, interactive dataset report: **https://github.com/gertelrina/aadd**
+* Backbone: Whisper-small encoder (88 M parameters), fine-tuned end to end
+* Output: one sigmoid per class per 20 ms frame on 8-s windows; any length via sliding windows
+* Languages seen in training: English and Russian
+
+## Usage
+
+```bash
+pip install "aadd[app] @ git+https://github.com/gertelrina/aadd"
+
+aadd scan my_dataset/ --model v3 --report report.html   # downloads this checkpoint on first use
+aadd scan call.wav --model v3 --resolution 0.1 --segments per_100ms.csv
+```
+
+```python
+from aadd import Detector
+
+det = Detector("v3", resolution=1.0)       # per-second summary; frame probabilities in res.frame_probs
+res = det.predict_file("call.wav")
+res.detected                                # {'noise': 0.97, 'reverb': 0.88}
+res.frame_probs.shape                       # (n_frames_20ms, 12)
+```
+
+Or download the file directly:
+
+```python
+from huggingface_hub import hf_hub_download
+path = hf_hub_download("agertel/aadd", "aadd_v3_whisper_small_frames.pt")
+det = Detector(path)
+```
+
+## Results
+
+**Out-of-domain, real recordings** (never seen in training). Clip score = 95th
+percentile of frame probabilities.
+
+| set | class | ROC-AUC | recall at p ≥ 0.5 | false alarms on clean |
+|---|---|---|---|---|
+| VoiceBank-DEMAND test, 824 noisy vs 824 clean twins | noise | 0.985 | 0.51 | 0.4 % |
+| DNS Challenge 2020 synthetic dev-test, 600 clips | noise | 0.997 | 0.85 | 0.0 % |
+| DNS Challenge 2020 synthetic dev-test, 600 clips | reverberation | 1.000 | 1.00 | 0.0 % |
+| Dynamic-SUPERB noise detection (VCTK + MUSAN) | noise | 0.995 | 0.96 | 1.1 % |
+| Dynamic-SUPERB reverberation, small / medium / large room | reverberation | 0.995 / 1.000 / 0.998 | 0.97 / 1.00 / 0.96 | 0 % |
+
+Ranking is near perfect; at the default threshold the model is conservative on
+mild noise (VoiceBank-DEMAND SNR 2.5–17.5 dB), so lower the threshold when
+filtering data.
+
+**Held-out synthetic test sets** (unseen speakers, noises and rooms; up to three
+overlapping degradations with random onsets):
+
+| test set | frame macro-AP (20 ms) | 1-s segment macro-F1 | clip macro-AUC |
+|---|---|---|---|
+| English (LibriSpeech test-clean), 3 000 × 8 s | 0.955 | 0.915 | 0.988 |
+| Russian (Russian LibriSpeech test), 3 000 × 8 s | 0.959 | 0.883 | 0.986 |
+| English, real measured RIRs | 0.951 | 0.909 | 0.989 |
+| English, MIT IR survey RIRs | 0.940 | 0.892 | 0.988 |
+
+Frame AP per class (English / Russian): pitch shift 0.999 / 0.992, time stretch
+0.997 / 0.992, clipping 0.996 / 0.996, high-pass 0.991 / 0.998, packet loss 0.989 /
+0.990, reverberation 0.987 / 0.989, music 0.983 / 0.985, clicks 0.972 / 0.975,
+low-pass 0.966 / 0.981, codec 0.920 / 0.949, background noise 0.885 / 0.803,
+mains hum 0.772 / 0.864.
+
+## Training
+
+* **Speech**: LibriSpeech train-clean-100 (English, 100 h) and Russian LibriSpeech
+  train (93 h); validation and test speakers are disjoint.
+* **Degradations**, generated on the fly in 8-s windows, 0–3 events per window,
+  whole-window or 0.5–4 s intervals with random onsets, fixed signal-chain order:
+  MUSAN noise (−5…30 dB SNR) and music (0…25 dB), RIRS_NOISES room impulse responses,
+  hard / soft clipping (2–30 % samples), Butterworth or resampling low-pass (1–5 kHz),
+  high-pass (200–1000 Hz), MP3 8–41 kbit/s, Opus 10–32 kbit/s, Vorbis, GSM, µ-law,
+  A-law, G.721, IMA-ADPCM, Gilbert–Elliott packet loss (5–30 %), 50/60 Hz hum,
+  clicks, pitch shift (±2–8 semitones), time stretch (0.7–0.9, 1.15–1.5).
+  Labels are per 20 ms; frames where a degradation is physically absent are masked.
+* **Model**: Whisper-small encoder, learnable weighted sum of all 13 hidden states,
+  two 1-D convolutions and a linear head, 12 sigmoids per frame.
+* **Recipe**: AdamW, lr 3e-5 (encoder) / 1e-3 (head), 1 000 warm-up steps, cosine,
+  bf16, batch 64 windows, 30 000 steps (≈ 4 300 h of generated audio), one H100,
+  2.4 h. Best validation checkpoint at step 16 000 (frame macro-AP 0.969).
+
+## Limitations
+
+* Degradations are synthetic. Real-world tests above cover noise and reverberation;
+  the other classes are validated only on held-out synthetic data.
+* The "clean" training speech comes from LibriVox audiobooks, which themselves
+  contain low-level noise, hum and MP3 history. The model therefore flags some of it
+  (14 % of English and 33 % of Russian synthetic "clean" windows have a frame above
+  0.5), which caps the measured AP for noise and hum. On studio-clean speech
+  (VoiceBank clean) the false-alarm rate is 0.4 %.
+* Read speech only; music, far-field meetings and TTS output were not evaluated.
+* Inputs are resampled to 16 kHz, so content above 8 kHz is not seen.
+
+## Licence and data
+
+Weights: MIT. Base model: Whisper-small (Apache 2.0). Training data: LibriSpeech
+(CC BY 4.0), Russian LibriSpeech (CC BY 4.0 card, public-domain source recordings),
+MUSAN (CC BY 4.0), RIRS_NOISES (Apache 2.0). Evaluation data: MIT IR survey,
+VoiceBank-DEMAND, DNS Challenge 2020, Dynamic-SUPERB.

aadd_v3_whisper_small_frames.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e0b335886c5d24442d56f323f628a96c2518f330f456e7b3792f82a64e4f6af
+size 177315122

eval_v3.json

@@ -0,0 +1,1413 @@
+{
+ "ckpt": "/home/agertel/work/aadd-runs/v3/aadd_v3_whisper_small_frames.pt",
+ "synthetic": {
+  "test_en": {
+   "frame": {
+    "per_class": {
+     "noise": {
+      "support": 78639,
+      "auc": 0.9739364880338869,
+      "ap": 0.8845758786765241,
+      "f1": 0.8733835217585184,
+      "precision": 0.9016830731317618,
+      "recall": 0.8468062920433881
+     },
+     "music": {
+      "support": 84000,
+      "auc": 0.9985092503206674,
+      "ap": 0.983359150076788,
+      "f1": 0.9658760647082152,
+      "precision": 0.9605146980916615,
+      "recall": 0.971297619047619
+     },
+     "reverb": {
+      "support": 78236,
+      "auc": 0.9960960632414917,
+      "ap": 0.9871238110008128,
+      "f1": 0.9512744200244202,
+      "precision": 0.9466030476383335,
+      "recall": 0.9559921263868296
+     },
+     "clipping": {
+      "support": 53342,
+      "auc": 0.9996628084076009,
+      "ap": 0.9958399399270511,
+      "f1": 0.9745924813152567,
+      "precision": 0.9701658895432,
+      "recall": 0.9790596528064189
+     },
+     "lowpass": {
+      "support": 103755,
+      "auc": 0.9972200477300588,
+      "ap": 0.9663775644557149,
+      "f1": 0.8828647210322427,
+      "precision": 0.7927302571025342,
+      "recall": 0.9961254879282926
+     },
+     "highpass": {
+      "support": 79278,
+      "auc": 0.9951698279030602,
+      "ap": 0.9914372288942742,
+      "f1": 0.9781819105432932,
+      "precision": 0.984886070122497,
+      "recall": 0.9715684048538056
+     },
+     "codec": {
+      "support": 83463,
+      "auc": 0.9844503000683807,
+      "ap": 0.9202691100421486,
+      "f1": 0.8556850330055235,
+      "precision": 0.804961505560308,
+      "recall": 0.9132310125444808
+     },
+     "packet_loss": {
+      "support": 8226,
+      "auc": 0.9991369577014577,
+      "ap": 0.9885315614558331,
+      "f1": 0.9582109239163019,
+      "precision": 0.9453448479829646,
+      "recall": 0.9714320447362023
+     },
+     "hum": {
+      "support": 86573,
+      "auc": 0.9747932322282458,
+      "ap": 0.7718321407856414,
+      "f1": 0.727107298368753,
+      "precision": 0.6409596502670591,
+      "recall": 0.8400078546429025
+     },
+     "clicks": {
+      "support": 8410,
+      "auc": 0.9970920712414454,
+      "ap": 0.9716312219485786,
+      "f1": 0.9391625615763548,
+      "precision": 0.9739463601532568,
+      "recall": 0.9067776456599287
+     },
+     "pitch_shift": {
+      "support": 63368,
+      "auc": 0.9999093360621429,
+      "ap": 0.9985917740639167,
+      "f1": 0.9807648859838421,
+      "precision": 0.9685187179763352,
+      "recall": 0.993324706476455
+     },
+     "time_stretch": {
+      "support": 69035,
+      "auc": 0.9997985179311355,
+      "ap": 0.9971987747664877,
+      "f1": 0.9749158831714806,
+      "precision": 0.9913158052344733,
+      "recall": 0.9590497573694503
+     }
+    },
+    "macro_auc": 0.9929812417391309,
+    "macro_ap": 0.9547306796744809,
+    "macro_f1": 0.9218349754503502
+   },
+   "segment_1s": {
+    "per_class": {
+     "noise": {
+      "support": 1795,
+      "auc": 0.9711004259498882,
+      "ap": 0.8866178416157908,
+      "f1": 0.876487252124646,
+      "precision": 0.89164265129683,
+      "recall": 0.8618384401114206
+     },
+     "music": {
+      "support": 1904,
+      "auc": 0.99813452489975,
+      "ap": 0.9821528246280575,
+      "f1": 0.9548755186721991,
+      "precision": 0.9431352459016393,
+      "recall": 0.9669117647058824
+     },
+     "reverb": {
+      "support": 1783,
+      "auc": 0.9944376847235489,
+      "ap": 0.9839126735628216,
+      "f1": 0.9413384742495181,
+      "precision": 0.9247835497835498,
+      "recall": 0.9584969153112731
+     },
+     "clipping": {
+      "support": 1701,
+      "auc": 0.9990176821176432,
+      "ap": 0.9942784478123448,
+      "f1": 0.9661164205039096,
+      "precision": 0.952054794520548,
+      "recall": 0.9805996472663139
+     },
+     "lowpass": {
+      "support": 2347,
+      "auc": 0.996268525655785,
+      "ap": 0.9597394004075738,
+      "f1": 0.8781402324709412,
+      "precision": 0.7840642785403414,
+      "recall": 0.9978696207925011
+     },
+     "highpass": {
+      "support": 1780,
+      "auc": 0.9956563198454678,
+      "ap": 0.9925368559180535,
+      "f1": 0.9734265734265733,
+      "precision": 0.9693593314763231,
+      "recall": 0.9775280898876404
+     },
+     "codec": {
+      "support": 1860,
+      "auc": 0.9821109314139737,
+      "ap": 0.9194277426543178,
+      "f1": 0.8442460317460317,
+      "precision": 0.783609576427256,
+      "recall": 0.9150537634408602
+     },
+     "packet_loss": {
+      "support": 851,
+      "auc": 0.9969630753304853,
+      "ap": 0.9810067056159665,
+      "f1": 0.9308390022675738,
+      "precision": 0.8992332968236583,
+      "recall": 0.9647473560517039
+     },
+     "hum": {
+      "support": 1900,
+      "auc": 0.974496713503215,
+      "ap": 0.7996367102653053,
+      "f1": 0.7360510134365748,
+      "precision": 0.648735447611401,
+      "recall": 0.8505263157894737
+     },
+     "clicks": {
+      "support": 1009,
+      "auc": 0.9895495994940707,
+      "ap": 0.9668417250577295,
+      "f1": 0.9308430085815244,
+      "precision": 0.948559670781893,
+      "recall": 0.9137760158572844
+     },
+     "pitch_shift": {
+      "support": 1568,
+      "auc": 0.9998322310602201,
+      "ap": 0.9974695657503819,
+      "f1": 0.9728549141965678,
+      "precision": 0.952351863164325,
+      "recall": 0.9942602040816326
+     },
+     "time_stretch": {
+      "support": 1687,
+      "auc": 0.9994617453027442,
+      "ap": 0.9947724673904705,
+      "f1": 0.9695704057279236,
+      "precision": 0.975975975975976,
+      "recall": 0.9632483698873741
+     }
+    },
+    "macro_auc": 0.9914191216080659,
+    "macro_ap": 0.9548660800565679,
+    "macro_f1": 0.9145657372836652
+   },
+   "clip": {
+    "per_class": {
+     "noise": {
+      "support": 367,
+      "auc": 0.9612723026023713,
+      "ap": 0.8973270908077903,
+      "f1": 0.898186889818689,
+      "precision": 0.92,
+      "recall": 0.8773841961852861
+     },
+     "music": {
+      "support": 377,
+      "auc": 0.9962224597546091,
+      "ap": 0.9782140352627451,
+      "f1": 0.9513797634691196,
+      "precision": 0.9427083333333334,
+      "recall": 0.9602122015915119
+     },
+     "reverb": {
+      "support": 358,
+      "auc": 0.990459762580405,
+      "ap": 0.9765173726916048,
+      "f1": 0.9348127600554786,
+      "precision": 0.928374655647383,
+      "recall": 0.9413407821229051
+     },
+     "clipping": {
+      "support": 349,
+      "auc": 0.9960489581160378,
+      "ap": 0.9914292731225425,
+      "f1": 0.9645390070921985,
+      "precision": 0.9550561797752809,
+      "recall": 0.9742120343839542
+     },
+     "lowpass": {
+      "support": 460,
+      "auc": 0.9947881718589523,
+      "ap": 0.963041972973026,
+      "f1": 0.907114624505929,
+      "precision": 0.8315217391304348,
+      "recall": 0.9978260869565218
+     },
+     "highpass": {
+      "support": 345,
+      "auc": 0.9970059226507274,
+      "ap": 0.9956927548894229,
+      "f1": 0.9798850574712644,
+      "precision": 0.9715099715099715,
+      "recall": 0.9884057971014493
+     },
+     "codec": {
+      "support": 358,
+      "auc": 0.9729646577207888,
+      "ap": 0.9201611173019183,
+      "f1": 0.8412698412698414,
+      "precision": 0.7989949748743719,
+      "recall": 0.888268156424581
+     },
+     "packet_loss": {
+      "support": 328,
+      "auc": 0.9932594795165766,
+      "ap": 0.9841272229654879,
+      "f1": 0.9392592592592591,
+      "precision": 0.9135446685878963,
+      "recall": 0.9664634146341463
+     },
+     "hum": {
+      "support": 348,
+      "auc": 0.9701607765121965,
+      "ap": 0.8548629199746827,
+      "f1": 0.7806788511749346,
+      "precision": 0.715311004784689,
+      "recall": 0.8591954022988506
+     },
+     "clicks": {
+      "support": 346,
+      "auc": 0.9821068427632411,
+      "ap": 0.9694729268045222,
+      "f1": 0.9543446244477173,
+      "precision": 0.972972972972973,
+      "recall": 0.9364161849710982
+     },
+     "pitch_shift": {
+      "support": 317,
+      "auc": 0.9998265748473566,
+      "ap": 0.9986047199260277,
+      "f1": 0.9752321981424148,
+      "precision": 0.9574468085106383,
+      "recall": 0.9936908517350158
+     },
+     "time_stretch": {
+      "support": 334,
+      "auc": 0.9992773268167341,
+      "ap": 0.9959707484069011,
+      "f1": 0.973134328358209,
+      "precision": 0.9702380952380952,
+      "recall": 0.9760479041916168
+     }
+    },
+    "macro_auc": 0.9877827696449996,
+    "macro_ap": 0.9604518462605559,
+    "macro_f1": 0.9249864337554211,
+    "false_alarm_rate_on_clean_clips": 0.14527027027027026,
+    "n_clean_clips": 592
+   },
+   "n_windows": 3000
+  },
+  "test_en_mit_rir": {
+   "frame": {
+    "per_class": {
+     "noise": {
+      "support": 27091,
+      "auc": 0.9762173424230487,
+      "ap": 0.8807451947330999,
+      "f1": 0.8734767506572761,
+      "precision": 0.9096650411703573,
+      "recall": 0.8400575836993835
+     },
+     "music": {
+      "support": 23749,
+      "auc": 0.9989217227607523,
+      "ap": 0.9792085617506755,
+      "f1": 0.969319330678606,
+      "precision": 0.9503196051460474,
+      "recall": 0.989094277653796
+     },
+     "reverb": {
+      "support": 25239,
+      "auc": 0.9517186542857926,
+      "ap": 0.8456230223549415,
+      "f1": 0.7919675805392679,
+      "precision": 0.9238854848933024,
+      "recall": 0.69301477871548
+     },
+     "clipping": {
+      "support": 17596,
+      "auc": 0.9996346875075283,
+      "ap": 0.9970022518461732,
+      "f1": 0.9732722634508348,
+      "precision": 0.9933136094674556,
+      "recall": 0.9540236417367584
+     },
+     "lowpass": {
+      "support": 29902,
+      "auc": 0.9943997404425854,
+      "ap": 0.9253195035948603,
+      "f1": 0.808141839705822,
+      "precision": 0.683429284872026,
+      "recall": 0.988529195371547
+     },
+     "highpass": {
+      "support": 28059,
+      "auc": 0.999647168662574,
+      "ap": 0.9964108961021396,
+      "f1": 0.9753624980131046,
+      "precision": 0.9667740354316925,
+      "recall": 0.9841049217719805
+     },
+     "codec": {
+      "support": 28632,
+      "auc": 0.9797695927079737,
+      "ap": 0.8707313470116367,
+      "f1": 0.7976521363832543,
+      "precision": 0.7886525791144642,
+      "recall": 0.8068594579491478
+     },
+     "packet_loss": {
+      "support": 2372,
+      "auc": 0.9995229322867298,
+      "ap": 0.9918031230962014,
+      "f1": 0.9691629955947136,
+      "precision": 0.964509394572025,
+      "recall": 0.9738617200674536
+     },
+     "hum": {
+      "support": 25147,
+      "auc": 0.9829957470865325,
+      "ap": 0.8257012439414378,
+      "f1": 0.7206877810168486,
+      "precision": 0.6150969918470621,
+      "recall": 0.8700441404541297
+     },
+     "clicks": {
+      "support": 2915,
+      "auc": 0.9984451215209421,
+      "ap": 0.9655776570931485,
+      "f1": 0.9219858156028369,
+      "precision": 0.9541284403669725,
+      "recall": 0.8919382504288165
+     },
+     "pitch_shift": {
+      "support": 24972,
+      "auc": 0.9998428919064356,
+      "ap": 0.9982855780946163,
+      "f1": 0.9781812390508043,
+      "precision": 0.9726049089469517,
+      "recall": 0.9838218805061669
+     },
+     "time_stretch": {
+      "support": 25199,
+      "auc": 0.9998748850987682,
+      "ap": 0.9983558341430309,
+      "f1": 0.980218982517694,
+      "precision": 0.9963925555464459,
+      "recall": 0.9645620857970555
+     }
+    },
+    "macro_auc": 0.9900825405574719,
+    "macro_ap": 0.9395636844801635,
+    "macro_f1": 0.896619101100922
+   },
+   "segment_1s": {
+    "per_class": {
+     "noise": {
+      "support": 613,
+      "auc": 0.9725175239514061,
+      "ap": 0.8823288132785112,
+      "f1": 0.8736401673640167,
+      "precision": 0.8969072164948454,
+      "recall": 0.8515497553017944
+     },
+     "music": {
+      "support": 546,
+      "auc": 0.9983796835487202,
+      "ap": 0.9795164670035255,
+      "f1": 0.9503546099290779,
+      "precision": 0.9209621993127147,
+      "recall": 0.9816849816849816
+     },
+     "reverb": {
+      "support": 578,
+      "auc": 0.9544013216109593,
+      "ap": 0.8572873024976929,
+      "f1": 0.8011417697431017,
+      "precision": 0.8900634249471459,
+      "recall": 0.7283737024221453
+     },
+     "clipping": {
+      "support": 556,
+      "auc": 0.9990006918473291,
+      "ap": 0.9948328491470319,
+      "f1": 0.9692028985507246,
+      "precision": 0.9762773722627737,
+      "recall": 0.9622302158273381
+     },
+     "lowpass": {
+      "support": 686,
+      "auc": 0.9926283592072713,
+      "ap": 0.9140178367107324,
+      "f1": 0.8056872037914693,
+      "precision": 0.6786427145708582,
+      "recall": 0.9912536443148688
+     },
+     "highpass": {
+      "support": 619,
+      "auc": 0.9996471751357403,
+      "ap": 0.9965799300279332,
+      "f1": 0.9553641346906812,
+      "precision": 0.9270516717325228,
+      "recall": 0.9854604200323102
+     },
+     "codec": {
+      "support": 640,
+      "auc": 0.9794826341711956,
+      "ap": 0.8842349402626255,
+      "f1": 0.8038490007401925,
+      "precision": 0.7637130801687764,
+      "recall": 0.8484375
+     },
+     "packet_loss": {
+      "support": 264,
+      "auc": 0.9993398422485272,
+      "ap": 0.9847558011438253,
+      "f1": 0.9239130434782609,
+      "precision": 0.8854166666666666,
+      "recall": 0.9659090909090909
+     },
+     "hum": {
+      "support": 556,
+      "auc": 0.9832676309248142,
+      "ap": 0.8476698707710674,
+      "f1": 0.730512249443207,
+      "precision": 0.6219974715549936,
+      "recall": 0.8848920863309353
+     },
+     "clicks": {
+      "support": 356,
+      "auc": 0.9948755530851453,
+      "ap": 0.9724774143866611,
+      "f1": 0.9321789321789322,
+      "precision": 0.9584569732937686,
+      "recall": 0.9073033707865169
+     },
+     "pitch_shift": {
+      "support": 597,
+      "auc": 0.9997101541748998,
+      "ap": 0.9976891887340611,
+      "f1": 0.9785831960461285,
+      "precision": 0.9627228525121556,
+      "recall": 0.9949748743718593
+     },
+     "time_stretch": {
+      "support": 603,
+      "auc": 0.9996322071316168,
+      "ap": 0.9953264715724265,
+      "f1": 0.9767441860465116,
+      "precision": 0.978369384359401,
+      "recall": 0.9751243781094527
+     }
+    },
+    "macro_auc": 0.9894068980864689,
+    "macro_ap": 0.9422264071280079,
+    "macro_f1": 0.8917642826668587
+   },
+   "clip": {
+    "per_class": {
+     "noise": {
+      "support": 124,
+      "auc": 0.9686119089703933,
+      "ap": 0.8998591966176134,
+      "f1": 0.8806584362139918,
+      "precision": 0.8991596638655462,
+      "recall": 0.8629032258064516
+     },
+     "music": {
+      "support": 114,
+      "auc": 0.9977971169458635,
+      "ap": 0.9843429612937049,
+      "f1": 0.9606986899563318,
+      "precision": 0.9565217391304348,
+      "recall": 0.9649122807017544
+     },
+     "reverb": {
+      "support": 121,
+      "auc": 0.9604499854267152,
+      "ap": 0.9032705541310309,
+      "f1": 0.8444444444444444,
+      "precision": 0.9134615384615384,
+      "recall": 0.7851239669421488
+     },
+     "clipping": {
+      "support": 112,
+      "auc": 0.9988990106177607,
+      "ap": 0.9946301217516003,
+      "f1": 0.9683257918552036,
+      "precision": 0.981651376146789,
+      "recall": 0.9553571428571429
+     },
+     "lowpass": {
+      "support": 145,
+      "auc": 0.9882153660012098,
+      "ap": 0.9135515716398611,
+      "f1": 0.8640483383685801,
+      "precision": 0.7688172043010753,
+      "recall": 0.9862068965517241
+     },
+     "highpass": {
+      "support": 118,
+      "auc": 0.9992553518582574,
+      "ap": 0.9954161978883156,
+      "f1": 0.9243027888446216,
+      "precision": 0.8721804511278195,
+      "recall": 0.9830508474576272
+     },
+     "codec": {
+      "support": 123,
+      "auc": 0.9719757858924085,
+      "ap": 0.9109254339034565,
+      "f1": 0.8136882129277566,
+      "precision": 0.7642857142857142,
+      "recall": 0.8699186991869918
+     },
+     "packet_loss": {
+      "support": 100,
+      "auc": 0.9991777777777777,
+      "ap": 0.994476221117894,
+      "f1": 0.9519230769230769,
+      "precision": 0.9166666666666666,
+      "recall": 0.99
+     },
+     "hum": {
+      "support": 106,
+      "auc": 0.9820923557469081,
+      "ap": 0.8810305301394162,
+      "f1": 0.7685950413223139,
+      "precision": 0.6838235294117647,
+      "recall": 0.8773584905660378
+     },
+     "clicks": {
+      "support": 122,
+      "auc": 0.9886291497068598,
+      "ap": 0.9800416309474747,
+      "f1": 0.9586776859504132,
+      "precision": 0.9666666666666667,
+      "recall": 0.9508196721311475
+     },
+     "pitch_shift": {
+      "support": 117,
+      "auc": 0.9976914365362837,
+      "ap": 0.9936060623589902,
+      "f1": 0.983050847457627,
+      "precision": 0.9747899159663865,
+      "recall": 0.9914529914529915
+     },
+     "time_stretch": {
+      "support": 121,
+      "auc": 0.9990879944339454,
+      "ap": 0.9959620833592652,
+      "f1": 0.9833333333333334,
+      "precision": 0.9915966386554622,
+      "recall": 0.9752066115702479
+     }
+    },
+    "macro_auc": 0.987656936659532,
+    "macro_ap": 0.9539260470957185,
+    "macro_f1": 0.9084788906331411,
+    "false_alarm_rate_on_clean_clips": 0.14864864864864866,
+    "n_clean_clips": 222
+   },
+   "n_windows": 1000
+  },
+  "test_en_real_rir": {
+   "frame": {
+    "per_class": {
+     "noise": {
+      "support": 24330,
+      "auc": 0.985375380387137,
+      "ap": 0.8874857361801259,
+      "f1": 0.868822582012376,
+      "precision": 0.8967976200892467,
+      "recall": 0.8425400739827373
+     },
+     "music": {
+      "support": 22023,
+      "auc": 0.9992015163280261,
+      "ap": 0.9832648899258126,
+      "f1": 0.960227523259403,
+      "precision": 0.9332333404756803,
+      "recall": 0.98882985969214
+     },
+     "reverb": {
+      "support": 27310,
+      "auc": 0.9791884060160176,
+      "ap": 0.9210360532135897,
+      "f1": 0.8887186438014419,
+      "precision": 0.9586388100182227,
+      "recall": 0.8283046503112413
+     },
+     "clipping": {
+      "support": 20978,
+      "auc": 0.999562147710874,
+      "ap": 0.9975167516789162,
+      "f1": 0.9762022223788381,
+      "precision": 0.9623882532771318,
+      "recall": 0.9904185337019735
+     },
+     "lowpass": {
+      "support": 40361,
+      "auc": 0.9956537932994404,
+      "ap": 0.9552235509514206,
+      "f1": 0.857934680651488,
+      "precision": 0.7528575437283697,
+      "recall": 0.9971011620128342
+     },
+     "highpass": {
+      "support": 20151,
+      "auc": 0.9998537508807454,
+      "ap": 0.9978010709585774,
+      "f1": 0.9752632747182282,
+      "precision": 0.9694057658364386,
+      "recall": 0.9811920003970026
+     },
+     "codec": {
+      "support": 25015,
+      "auc": 0.988396873468717,
+      "ap": 0.9176298609670425,
+      "f1": 0.8526378896882494,
+      "precision": 0.8524675324675325,
+      "recall": 0.8528083150109934
+     },
+     "packet_loss": {
+      "support": 3157,
+      "auc": 0.9989400567205612,
+      "ap": 0.9915215457352285,
+      "f1": 0.9641622131405218,
+      "precision": 0.9569422776911076,
+      "recall": 0.9714919227114349
+     },
+     "hum": {
+      "support": 26535,
+      "auc": 0.9846001230266842,
+      "ap": 0.7862753581481026,
+      "f1": 0.7851138873323006,
+      "precision": 0.6976771434429831,
+      "recall": 0.8976069342377991
+     },
+     "clicks": {
+      "support": 2587,
+      "auc": 0.9992228126586093,
+      "ap": 0.9852101080028043,
+      "f1": 0.9604766633565045,
+      "precision": 0.9877450980392157,
+      "recall": 0.9346733668341709
+     },
+     "pitch_shift": {
+      "support": 27914,
+      "auc": 0.99961636190629,
+      "ap": 0.9957944890414314,
+      "f1": 0.9652028672837358,
+      "precision": 0.9382991678506191,
+      "recall": 0.9936949201117719
+     },
+     "time_stretch": {
+      "support": 27908,
+      "auc": 0.9993907723776294,
+      "ap": 0.9934888998249539,
+      "f1": 0.9613111131571678,
+      "precision": 0.9888244876311703,
+      "recall": 0.9352873727963308
+     }
+    },
+    "macro_auc": 0.9940834995650608,
+    "macro_ap": 0.9510206928856669,
+    "macro_f1": 0.9180061300650212
+   },
+   "segment_1s": {
+    "per_class": {
+     "noise": {
+      "support": 569,
+      "auc": 0.9837713762159613,
+      "ap": 0.8944177474876214,
+      "f1": 0.875222816399287,
+      "precision": 0.8878842676311031,
+      "recall": 0.8629173989455184
+     },
+     "music": {
+      "support": 504,
+      "auc": 0.9991875296454404,
+      "ap": 0.9861260616177089,
+      "f1": 0.9529298751200769,
+      "precision": 0.9236499068901304,
+      "recall": 0.9841269841269841
+     },
+     "reverb": {
+      "support": 611,
+      "auc": 0.9799173983355185,
+      "ap": 0.9255989629525935,
+      "f1": 0.8786324786324786,
+      "precision": 0.9194991055456172,
+      "recall": 0.8412438625204582
+     },
+     "clipping": {
+      "support": 650,
+      "auc": 0.9992025934305346,
+      "ap": 0.9953779073640093,
+      "f1": 0.9609609609609611,
+      "precision": 0.9384164222873901,
+      "recall": 0.9846153846153847
+     },
+     "lowpass": {
+      "support": 880,
+      "auc": 0.9945571852655771,
+      "ap": 0.9477433394346417,
+      "f1": 0.8459681313375181,
+      "precision": 0.7355163727959698,
+      "recall": 0.9954545454545455
+     },
+     "highpass": {
+      "support": 459,
+      "auc": 0.9999396183940283,
+      "ap": 0.9990306890769586,
+      "f1": 0.9733191035218784,
+      "precision": 0.9539748953974896,
+      "recall": 0.9934640522875817
+     },
+     "codec": {
+      "support": 555,
+      "auc": 0.9823923184433594,
+      "ap": 0.9128917841982765,
+      "f1": 0.8380281690140845,
+      "precision": 0.8192771084337349,
+      "recall": 0.8576576576576577
+     },
+     "packet_loss": {
+      "support": 312,
+      "auc": 0.9963231113697397,
+      "ap": 0.9817961447999888,
+      "f1": 0.926605504587156,
+      "precision": 0.8859649122807017,
+      "recall": 0.9711538461538461
+     },
+     "hum": {
+      "support": 592,
+      "auc": 0.9831309103379838,
+      "ap": 0.811450998903163,
+      "f1": 0.7810650887573964,
+      "precision": 0.6947368421052632,
+      "recall": 0.8918918918918919
+     },
+     "clicks": {
+      "support": 292,
+      "auc": 0.9994819472385923,
+      "ap": 0.9892621761486452,
+      "f1": 0.9550173010380623,
+      "precision": 0.965034965034965,
+      "recall": 0.9452054794520548
+     },
+     "pitch_shift": {
+      "support": 661,
+      "auc": 0.9996374002567264,
+      "ap": 0.996298854994608,
+      "f1": 0.9611151870873075,
+      "precision": 0.9330484330484331,
+      "recall": 0.9909228441754917
+     },
+     "time_stretch": {
+      "support": 679,
+      "auc": 0.998992246767676,
+      "ap": 0.991084587178196,
+      "f1": 0.9548192771084337,
+      "precision": 0.9768875192604006,
+      "recall": 0.9337260677466863
+     }
+    },
+    "macro_auc": 0.9930444696417616,
+    "macro_ap": 0.9525899378463677,
+    "macro_f1": 0.90864032446372
+   },
+   "clip": {
+    "per_class": {
+     "noise": {
+      "support": 124,
+      "auc": 0.972183863602887,
+      "ap": 0.9033236001845246,
+      "f1": 0.8934426229508197,
+      "precision": 0.9083333333333333,
+      "recall": 0.8790322580645161
+     },
+     "music": {
+      "support": 102,
+      "auc": 0.9978328747980261,
+      "ap": 0.9848134369590466,
+      "f1": 0.9473684210526315,
+      "precision": 0.9252336448598131,
+      "recall": 0.9705882352941176
+     },
+     "reverb": {
+      "support": 119,
+      "auc": 0.982954816432816,
+      "ap": 0.9464112503272591,
+      "f1": 0.8831168831168831,
+      "precision": 0.9107142857142857,
+      "recall": 0.8571428571428571
+     },
+     "clipping": {
+      "support": 114,
+      "auc": 0.9942279513682628,
+      "ap": 0.9871795242797485,
+      "f1": 0.9523809523809523,
+      "precision": 0.9401709401709402,
+      "recall": 0.9649122807017544
+     },
+     "lowpass": {
+      "support": 159,
+      "auc": 0.9921439735564879,
+      "ap": 0.9427512643159155,
+      "f1": 0.8681318681318682,
+      "precision": 0.7707317073170732,
+      "recall": 0.9937106918238994
+     },
+     "highpass": {
+      "support": 90,
+      "auc": 0.99997557997558,
+      "ap": 0.9997584541062802,
+      "f1": 0.972972972972973,
+      "precision": 0.9473684210526315,
+      "recall": 1.0
+     },
+     "codec": {
+      "support": 104,
+      "auc": 0.9661315247252747,
+      "ap": 0.9051715964069142,
+      "f1": 0.8169014084507041,
+      "precision": 0.7981651376146789,
+      "recall": 0.8365384615384616
+     },
+     "packet_loss": {
+      "support": 126,
+      "auc": 0.988562892739094,
+      "ap": 0.9823675208411106,
+      "f1": 0.9448818897637795,
+      "precision": 0.9375,
+      "recall": 0.9523809523809523
+     },
+     "hum": {
+      "support": 113,
+      "auc": 0.9748929971765222,
+      "ap": 0.8456651771224659,
+      "f1": 0.8016528925619835,
+      "precision": 0.751937984496124,
+      "recall": 0.8584070796460177
+     },
+     "clicks": {
+      "support": 101,
+      "auc": 0.9996695998854612,
+      "ap": 0.9973576751073859,
+      "f1": 0.99,
+      "precision": 1.0,
+      "recall": 0.9801980198019802
+     },
+     "pitch_shift": {
+      "support": 127,
+      "auc": 0.9994813792605821,
+      "ap": 0.9965433185604732,
+      "f1": 0.9652509652509653,
+      "precision": 0.946969696969697,
+      "recall": 0.984251968503937
+     },
+     "time_stretch": {
+      "support": 128,
+      "auc": 0.998709862385321,
+      "ap": 0.9930289520341448,
+      "f1": 0.9609375,
+      "precision": 0.9609375,
+      "recall": 0.9609375
+     }
+    },
+    "macro_auc": 0.9888972763255262,
+    "macro_ap": 0.9570309808537725,
+    "macro_f1": 0.9164198647194634,
+    "false_alarm_rate_on_clean_clips": 0.12935323383084577,
+    "n_clean_clips": 201
+   },
+   "n_windows": 1000
+  },
+  "test_ru": {
+   "frame": {
+    "per_class": {
+     "noise": {
+      "support": 81329,
+      "auc": 0.9604987261124202,
+      "ap": 0.8025277031107889,
+      "f1": 0.5618683718028696,
+      "precision": 0.4113821788057162,
+      "recall": 0.8859570386946846
+     },
+     "music": {
+      "support": 80682,
+      "auc": 0.9985034750549702,
+      "ap": 0.9854590810432485,
+      "f1": 0.9546264162030151,
+      "precision": 0.9430074369671685,
+      "recall": 0.966535286681044
+     },
+     "reverb": {
+      "support": 85886,
+      "auc": 0.9975715969387704,
+      "ap": 0.9890707602861478,
+      "f1": 0.9639349742989678,
+      "precision": 0.9891444184422363,
+      "recall": 0.9399785762522413
+     },
+     "clipping": {
+      "support": 45997,
+      "auc": 0.9995284939468562,
+      "ap": 0.996160535403624,
+      "f1": 0.9809907707218221,
+      "precision": 0.9940849534608602,
+      "recall": 0.9682370589386264
+     },
+     "lowpass": {
+      "support": 111106,
+      "auc": 0.9980785319290724,
+      "ap": 0.9809664204069385,
+      "f1": 0.8462654958084246,
+      "precision": 0.7365228346876604,
+      "recall": 0.9944377441362303
+     },
+     "highpass": {
+      "support": 83549,
+      "auc": 0.9995308571986256,
+      "ap": 0.9975041507800847,
+      "f1": 0.9891165907101681,
+      "precision": 0.9977470894831701,
+      "recall": 0.9806341189002861
+     },
+     "codec": {
+      "support": 80596,
+      "auc": 0.9903866937271524,
+      "ap": 0.9493959330701672,
+      "f1": 0.8593606228618615,
+      "precision": 0.8190322000359777,
+      "recall": 0.9038661968335897
+     },
+     "packet_loss": {
+      "support": 8257,
+      "auc": 0.9991328330939298,
+      "ap": 0.9896103067689855,
+      "f1": 0.9723499478879284,
+      "precision": 0.9846039235162652,
+      "recall": 0.960397238706552
+     },
+     "hum": {
+      "support": 77260,
+      "auc": 0.9700310117315668,
+      "ap": 0.8643516822737889,
+      "f1": 0.808859552219348,
+      "precision": 0.8607058892247484,
+      "recall": 0.7629044783846751
+     },
+     "clicks": {
+      "support": 8383,
+      "auc": 0.9974901436189798,
+      "ap": 0.9751135933448568,
+      "f1": 0.948892570096325,
+      "precision": 0.9768822637695806,
+      "recall": 0.9224621257306453
+     },
+     "pitch_shift": {
+      "support": 72566,
+      "auc": 0.999441627327891,
+      "ap": 0.9921328301966863,
+      "f1": 0.9513541590362729,
+      "precision": 0.9204907153258334,
+      "recall": 0.9843590662293636
+     },
+     "time_stretch": {
+      "support": 70396,
+      "auc": 0.9992112838509106,
+      "ap": 0.991643830142524,
+      "f1": 0.9565343127331617,
+      "precision": 0.9739693757361602,
+      "recall": 0.9397124836638445
+     }
+    },
+    "macro_auc": 0.9924504395442622,
+    "macro_ap": 0.9594947355689866,
+    "macro_f1": 0.8995128153650137
+   },
+   "segment_1s": {
+    "per_class": {
+     "noise": {
+      "support": 1826,
+      "auc": 0.9588785416270065,
+      "ap": 0.8118856763812923,
+      "f1": 0.5483116014710799,
+      "precision": 0.3946102021174206,
+      "recall": 0.8981380065717415
+     },
+     "music": {
+      "support": 1838,
+      "auc": 0.9979314453594704,
+      "ap": 0.9842554962249991,
+      "f1": 0.9466382070437568,
+      "precision": 0.9287958115183246,
+      "recall": 0.9651795429815017
+     },
+     "reverb": {
+      "support": 1938,
+      "auc": 0.9965777091841466,
+      "ap": 0.9869481133356713,
+      "f1": 0.9584313725490198,
+      "precision": 0.9713831478537361,
+      "recall": 0.9458204334365325
+     },
+     "clipping": {
+      "support": 1581,
+      "auc": 0.9990370634324373,
+      "ap": 0.9931273281736334,
+      "f1": 0.9778200253485425,
+      "precision": 0.9796825396825397,
+      "recall": 0.9759645793801391
+     },
+     "lowpass": {
+      "support": 2495,
+      "auc": 0.9972973149274601,
+      "ap": 0.9772449244467835,
+      "f1": 0.8025210084033614,
+      "precision": 0.6723531004603304,
+      "recall": 0.995190380761523
+     },
+     "highpass": {
+      "support": 1879,
+      "auc": 0.9996662846097396,
+      "ap": 0.9978072300464331,
+      "f1": 0.9874230666309874,
+      "precision": 0.9930032292787944,
+      "recall": 0.9819052687599787
+     },
+     "codec": {
+      "support": 1816,
+      "auc": 0.9865570281434729,
+      "ap": 0.9429442150035149,
+      "f1": 0.8288016112789527,
+      "precision": 0.7634508348794063,
+      "recall": 0.9063876651982379
+     },
+     "packet_loss": {
+      "support": 872,
+      "auc": 0.9975980890461252,
+      "ap": 0.9839538081721221,
+      "f1": 0.9534215066129961,
+      "precision": 0.9561707035755479,
+      "recall": 0.9506880733944955
+     },
+     "hum": {
+      "support": 1735,
+      "auc": 0.967002344693957,
+      "ap": 0.8588409405361159,
+      "f1": 0.7922962962962962,
+      "precision": 0.8152439024390243,
+      "recall": 0.7706051873198847
+     },
+     "clicks": {
+      "support": 982,
+      "auc": 0.9935183007375724,
+      "ap": 0.9668101783472817,
+      "f1": 0.9295336787564766,
+      "precision": 0.9462025316455697,
+      "recall": 0.9134419551934827
+     },
+     "pitch_shift": {
+      "support": 1748,
+      "auc": 0.999050617322421,
+      "ap": 0.9902967087244904,
+      "f1": 0.9265141318977119,
+      "precision": 0.874936451448907,
+      "recall": 0.9845537757437071
+     },
+     "time_stretch": {
+      "support": 1710,
+      "auc": 0.9983450389474368,
+      "ap": 0.9868860344179521,
+      "f1": 0.9496946786856645,
+      "precision": 0.9444765760555234,
+      "recall": 0.9549707602339181
+     }
+    },
+    "macro_auc": 0.9909549815026036,
+    "macro_ap": 0.9567500544841908,
+    "macro_f1": 0.8834505987479039
+   },
+   "clip": {
+    "per_class": {
+     "noise": {
+      "support": 358,
+      "auc": 0.9529680621164769,
+      "ap": 0.8369869489774066,
+      "f1": 0.6013071895424836,
+      "precision": 0.45161290322580644,
+      "recall": 0.8994413407821229
+     },
+     "music": {
+      "support": 369,
+      "auc": 0.995261315212924,
+      "ap": 0.9817835501318576,
+      "f1": 0.9503355704697988,
+      "precision": 0.9414893617021277,
+      "recall": 0.959349593495935
+     },
+     "reverb": {
+      "support": 383,
+      "auc": 0.9873681921080383,
+      "ap": 0.9763234791904579,
+      "f1": 0.949868073878628,
+      "precision": 0.96,
+      "recall": 0.9399477806788512
+     },
+     "clipping": {
+      "support": 331,
+      "auc": 0.9964021285001002,
+      "ap": 0.9920111759404076,
+      "f1": 0.9802130898021308,
+      "precision": 0.9877300613496932,
+      "recall": 0.972809667673716
+     },
+     "lowpass": {
+      "support": 474,
+      "auc": 0.9939364783467133,
+      "ap": 0.9724250775619331,
+      "f1": 0.8156521739130436,
+      "precision": 0.6937869822485208,
+      "recall": 0.989451476793249
+     },
+     "highpass": {
+      "support": 365,
+      "auc": 0.99965116581321,
+      "ap": 0.9985539176738065,
+      "f1": 0.9931412894375857,
+      "precision": 0.9945054945054945,
+      "recall": 0.9917808219178083
+     },
+     "codec": {
+      "support": 351,
+      "auc": 0.9704048939609529,
+      "ap": 0.9252290019010437,
+      "f1": 0.8072445019404916,
+      "precision": 0.7393364928909952,
+      "recall": 0.8888888888888888
+     },
+     "packet_loss": {
+      "support": 329,
+      "auc": 0.9959050206029184,
+      "ap": 0.9901403382160633,
+      "f1": 0.9679389312977099,
+      "precision": 0.9723926380368099,
+      "recall": 0.9635258358662614
+     },
+     "hum": {
+      "support": 350,
+      "auc": 0.952677628032345,
+      "ap": 0.857516983844412,
+      "f1": 0.7904903417533432,
+      "precision": 0.8235294117647058,
+      "recall": 0.76
+     },
+     "clicks": {
+      "support": 333,
+      "auc": 0.9925251460684531,
+      "ap": 0.9816671501584321,
+      "f1": 0.9586523736600306,
+      "precision": 0.978125,
+      "recall": 0.93993993993994
+     },
+     "pitch_shift": {
+      "support": 348,
+      "auc": 0.9980311974480331,
+      "ap": 0.9896868976831013,
+      "f1": 0.9193548387096775,
+      "precision": 0.8636363636363636,
+      "recall": 0.9827586206896551
+     },
+     "time_stretch": {
+      "support": 343,
+      "auc": 0.9946189777593922,
+      "ap": 0.983387683415535,
+      "f1": 0.9416785206258891,
+      "precision": 0.9194444444444444,
+      "recall": 0.9650145772594753
+     }
+    },
+    "macro_auc": 0.9858125171641299,
+    "macro_ap": 0.957142683724538,
+    "macro_f1": 0.8896564079192344,
+    "false_alarm_rate_on_clean_clips": 0.33220910623946037,
+    "n_clean_clips": 593
+   },
+   "n_windows": 3000
+  }
+ },
+ "ood": {
+  "VoiceBank-DEMAND test (824 noisy vs 824 clean twins)": {
+   "n": 1648,
+   "noise": {
+    "auc_mean": 0.9891233268922612,
+    "auc_p95": 0.9854766589688,
+    "recall_at_0.5_p95": 0.5060679611650486,
+    "false_alarm_at_0.5_p95": 0.0036407766990291263
+   },
+   "clean_clips_any_class_false_alarm": 0.0036407766990291263,
+   "clean_clips_false_alarm_by_class": {
+    "noise": 0.0036407766990291263,
+    "music": 0.0,
+    "reverb": 0.0,
+    "clipping": 0.0,
+    "lowpass": 0.0,
+    "highpass": 0.0,
+    "codec": 0.0,
+    "packet_loss": 0.0,
+    "hum": 0.0,
+    "clicks": 0.0,
+    "pitch_shift": 0.0,
+    "time_stretch": 0.0
+   }
+  },
+  "DNS-2020 synthetic (clean / noisy, dry / reverberant)": {
+   "n": 600,
+   "noise": {
+    "auc_mean": 0.9989111111111111,
+    "auc_p95": 0.9973333333333333,
+    "recall_at_0.5_p95": 0.8533333333333334,
+    "false_alarm_at_0.5_p95": 0.0
+   },
+   "reverb": {
+    "auc_mean": 1.0,
+    "auc_p95": 1.0,
+    "recall_at_0.5_p95": 1.0,
+    "false_alarm_at_0.5_p95": 0.0
+   },
+   "clean_clips_any_class_false_alarm": 0.006666666666666667,
+   "clean_clips_false_alarm_by_class": {
+    "noise": 0.0,
+    "music": 0.0,
+    "reverb": 0.0,
+    "clipping": 0.0,
+    "lowpass": 0.0,
+    "highpass": 0.0,
+    "codec": 0.006666666666666667,
+    "packet_loss": 0.0,
+    "hum": 0.0,
+    "clicks": 0.0,
+    "pitch_shift": 0.0,
+    "time_stretch": 0.0
+   }
+  },
+  "Dynamic-SUPERB NoiseDetection_VCTK_MUSAN-Noise": {
+   "n": 200,
+   "noise": {
+    "auc_mean": 0.9961689686460329,
+    "auc_p95": 0.994959169271096,
+    "recall_at_0.5_p95": 0.963302752293578,
+    "false_alarm_at_0.5_p95": 0.01098901098901099
+   },
+   "clean_clips_any_class_false_alarm": 0.03296703296703297,
+   "clean_clips_false_alarm_by_class": {
+    "noise": 0.01098901098901099,
+    "music": 0.0,
+    "reverb": 0.0,
+    "clipping": 0.0,
+    "lowpass": 0.0,
+    "highpass": 0.0,
+    "codec": 0.0,
+    "packet_loss": 0.0,
+    "hum": 0.0,
+    "clicks": 0.0,
+    "pitch_shift": 0.0,
+    "time_stretch": 0.02197802197802198
+   }
+  },
+  "Dynamic-SUPERB ReverberationDetection_VCTK_RirsNoises-SmallRoom": {
+   "n": 200,
+   "reverb": {
+    "auc_mean": 0.9962761674718197,
+    "auc_p95": 0.9948671497584541,
+    "recall_at_0.5_p95": 0.967391304347826,
+    "false_alarm_at_0.5_p95": 0.0
+   },
+   "clean_clips_any_class_false_alarm": 0.06481481481481481,
+   "clean_clips_false_alarm_by_class": {
+    "noise": 0.018518518518518517,
+    "music": 0.0,
+    "reverb": 0.0,
+    "clipping": 0.0,
+    "lowpass": 0.0,
+    "highpass": 0.0,
+    "codec": 0.0,
+    "packet_loss": 0.0,
+    "hum": 0.0,
+    "clicks": 0.0,
+    "pitch_shift": 0.0,
+    "time_stretch": 0.046296296296296294
+   }
+  },
+  "Dynamic-SUPERB ReverberationDetection_VCTK_RirsNoises-MediumRoom": {
+   "n": 200,
+   "reverb": {
+    "auc_mean": 1.0,
+    "auc_p95": 1.0,
+    "recall_at_0.5_p95": 1.0,
+    "false_alarm_at_0.5_p95": 0.0
+   },
+   "clean_clips_any_class_false_alarm": 0.06481481481481481,
+   "clean_clips_false_alarm_by_class": {
+    "noise": 0.018518518518518517,
+    "music": 0.0,
+    "reverb": 0.0,
+    "clipping": 0.0,
+    "lowpass": 0.0,
+    "highpass": 0.0,
+    "codec": 0.0,
+    "packet_loss": 0.0,
+    "hum": 0.0,
+    "clicks": 0.0,
+    "pitch_shift": 0.0,
+    "time_stretch": 0.046296296296296294
+   }
+  },
+  "Dynamic-SUPERB ReverberationDetection_VCTK_RirsNoises-LargeRoom": {
+   "n": 200,
+   "reverb": {
+    "auc_mean": 0.9980877616747181,
+    "auc_p95": 0.9976348631239936,
+    "recall_at_0.5_p95": 0.9565217391304348,
+    "false_alarm_at_0.5_p95": 0.0
+   },
+   "clean_clips_any_class_false_alarm": 0.05555555555555555,
+   "clean_clips_false_alarm_by_class": {
+    "noise": 0.018518518518518517,
+    "music": 0.0,
+    "reverb": 0.0,
+    "clipping": 0.0,
+    "lowpass": 0.0,
+    "highpass": 0.0,
+    "codec": 0.0,
+    "packet_loss": 0.0,
+    "hum": 0.0,
+    "clicks": 0.0,
+    "pitch_shift": 0.0,
+    "time_stretch": 0.037037037037037035
+   }
+  }
+ }
+}

v3_whisper_small_log.json

@@ -0,0 +1,1070 @@
+{
+ "history": [
+  {
+   "step": 2000,
+   "loss": 0.06958092003222555,
+   "val_macro_ap": 0.9326372090429093,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9876570199702029,
+      "macro_ap": 0.9223619137093909,
+      "macro_f1": 0.8696549891166528
+     },
+     "segment_1s": {
+      "macro_auc": 0.9850394194980091,
+      "macro_ap": 0.9225303106157062,
+      "macro_f1": 0.8625755323616674
+     },
+     "clip": {
+      "macro_auc": 0.9778469423338437,
+      "macro_ap": 0.9296743856374521,
+      "macro_f1": 0.8676454685360909
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9884696827841172,
+      "macro_ap": 0.9429125043764276,
+      "macro_f1": 0.8839522573463313
+     },
+     "segment_1s": {
+      "macro_auc": 0.9854449505770501,
+      "macro_ap": 0.9425857879601494,
+      "macro_f1": 0.8753004424994777
+     },
+     "clip": {
+      "macro_auc": 0.9767192220193359,
+      "macro_ap": 0.9436841360926899,
+      "macro_f1": 0.8761214464133652
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8554073435525561,
+     "music": 0.9708626825534987,
+     "reverb": 0.9545400645738038,
+     "clipping": 0.968802831615579,
+     "lowpass": 0.9943552722748055,
+     "highpass": 0.9878460926654185,
+     "codec": 0.6929632004212506,
+     "packet_loss": 0.968094637440758,
+     "hum": 0.7824939891534918,
+     "clicks": 0.9470210210029005,
+     "pitch_shift": 0.9593597959224034,
+     "time_stretch": 0.9865960333362271
+    },
+    "val_ru": {
+     "noise": 0.9003590486188318,
+     "music": 0.9773023551103137,
+     "reverb": 0.9771296304150066,
+     "clipping": 0.9834707505068949,
+     "lowpass": 0.9950073012257871,
+     "highpass": 0.9985482235069165,
+     "codec": 0.6804421629789246,
+     "packet_loss": 0.9605571129609488,
+     "hum": 0.9309695085436116,
+     "clicks": 0.9411477577835741,
+     "pitch_shift": 0.9828225566651935,
+     "time_stretch": 0.9871936442011285
+    }
+   }
+  },
+  {
+   "step": 4000,
+   "loss": 0.02800460657197982,
+   "val_macro_ap": 0.9498887181691418,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9915750453200999,
+      "macro_ap": 0.9325498416387247,
+      "macro_f1": 0.8989665890705965
+     },
+     "segment_1s": {
+      "macro_auc": 0.9893142359011132,
+      "macro_ap": 0.9297670463040605,
+      "macro_f1": 0.8905066149181771
+     },
+     "clip": {
+      "macro_auc": 0.9838457654309342,
+      "macro_ap": 0.9348963195325265,
+      "macro_f1": 0.8992158747694369
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9930155810161447,
+      "macro_ap": 0.9672275946995589,
+      "macro_f1": 0.9065461292286902
+     },
+     "segment_1s": {
+      "macro_auc": 0.9903622778403709,
+      "macro_ap": 0.9651228617525002,
+      "macro_f1": 0.8921051370100851
+     },
+     "clip": {
+      "macro_auc": 0.9814589944788598,
+      "macro_ap": 0.9588180552443335,
+      "macro_f1": 0.8842346441198634
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.817017095027065,
+     "music": 0.9770009303219283,
+     "reverb": 0.9635767534250257,
+     "clipping": 0.9878455085080631,
+     "lowpass": 0.9968923331803484,
+     "highpass": 0.9844621243526334,
+     "codec": 0.6307590652545713,
+     "packet_loss": 0.977974799616703,
+     "hum": 0.9268411117549413,
+     "clicks": 0.9547330711151069,
+     "pitch_shift": 0.9852052328831601,
+     "time_stretch": 0.9882900742251477
+    },
+    "val_ru": {
+     "noise": 0.927132158041044,
+     "music": 0.9858739868711583,
+     "reverb": 0.9890132279163512,
+     "clipping": 0.9681573929363416,
+     "lowpass": 0.9972631332323754,
+     "highpass": 0.9988967978640078,
+     "codec": 0.8742254643230409,
+     "packet_loss": 0.9653213784758006,
+     "hum": 0.964290826971642,
+     "clicks": 0.959633727258145,
+     "pitch_shift": 0.9861971773835936,
+     "time_stretch": 0.9907258651212085
+    }
+   }
+  },
+  {
+   "step": 6000,
+   "loss": 0.023227403885452077,
+   "val_macro_ap": 0.9538960576294403,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9918893274535386,
+      "macro_ap": 0.9362371450444998,
+      "macro_f1": 0.8991431370053499
+     },
+     "segment_1s": {
+      "macro_auc": 0.9900127525959338,
+      "macro_ap": 0.9343538006046366,
+      "macro_f1": 0.8907171390877743
+     },
+     "clip": {
+      "macro_auc": 0.9855407766939233,
+      "macro_ap": 0.9408338535839311,
+      "macro_f1": 0.9012862473270647
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9938431663145114,
+      "macro_ap": 0.9715549702143808,
+      "macro_f1": 0.9276126191547495
+     },
+     "segment_1s": {
+      "macro_auc": 0.9914274840266889,
+      "macro_ap": 0.9676084829212818,
+      "macro_f1": 0.9107859045027418
+     },
+     "clip": {
+      "macro_auc": 0.9846560496071427,
+      "macro_ap": 0.9642101628715937,
+      "macro_f1": 0.9031423298399085
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.9041335719957007,
+     "music": 0.981361759363393,
+     "reverb": 0.9552409523380546,
+     "clipping": 0.9858502585660558,
+     "lowpass": 0.9985991920921061,
+     "highpass": 0.9792205934142189,
+     "codec": 0.5672542697194355,
+     "packet_loss": 0.9892736219612331,
+     "hum": 0.9327577258064714,
+     "clicks": 0.9604496853461502,
+     "pitch_shift": 0.9908502305737855,
+     "time_stretch": 0.9898538793573937
+    },
+    "val_ru": {
+     "noise": 0.9051812171277394,
+     "music": 0.9872088864970957,
+     "reverb": 0.9747772030884757,
+     "clipping": 0.9949316796795749,
+     "lowpass": 0.9972250388985557,
+     "highpass": 0.999517218249077,
+     "codec": 0.9091552725590806,
+     "packet_loss": 0.9763664063401016,
+     "hum": 0.9655093626407706,
+     "clicks": 0.9721004744067954,
+     "pitch_shift": 0.9820270793002209,
+     "time_stretch": 0.9946598037850828
+    }
+   }
+  },
+  {
+   "step": 8000,
+   "loss": 0.02032090468192473,
+   "val_macro_ap": 0.9541761854077329,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.993224437463184,
+      "macro_ap": 0.9367533451154194,
+      "macro_f1": 0.8900909880994033
+     },
+     "segment_1s": {
+      "macro_auc": 0.9911032589519452,
+      "macro_ap": 0.9355060679972961,
+      "macro_f1": 0.883454683819734
+     },
+     "clip": {
+      "macro_auc": 0.9867800080672096,
+      "macro_ap": 0.9431908870141362,
+      "macro_f1": 0.8980695992558263
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9942923101247869,
+      "macro_ap": 0.9715990257000464,
+      "macro_f1": 0.9317793615383553
+     },
+     "segment_1s": {
+      "macro_auc": 0.9918480367888071,
+      "macro_ap": 0.9668836488393072,
+      "macro_f1": 0.9153864303809774
+     },
+     "clip": {
+      "macro_auc": 0.9850874644415099,
+      "macro_ap": 0.9625832508752015,
+      "macro_f1": 0.9109409436834897
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8612098135362084,
+     "music": 0.9819620822744896,
+     "reverb": 0.9709252816234836,
+     "clipping": 0.9815754752233243,
+     "lowpass": 0.9976472393437494,
+     "highpass": 0.9561357826765595,
+     "codec": 0.6268571277017085,
+     "packet_loss": 0.988532211579522,
+     "hum": 0.9254097856865139,
+     "clicks": 0.968291140100442,
+     "pitch_shift": 0.9884887239403438,
+     "time_stretch": 0.9940054776986865
+    },
+    "val_ru": {
+     "noise": 0.9391879528053555,
+     "music": 0.9846558454312485,
+     "reverb": 0.9754690821709301,
+     "clipping": 0.9988049867154266,
+     "lowpass": 0.9952321257344624,
+     "highpass": 0.9996190018541707,
+     "codec": 0.8561816789321227,
+     "packet_loss": 0.9777821023120562,
+     "hum": 0.9736098532364756,
+     "clicks": 0.9737336957883805,
+     "pitch_shift": 0.9887540857446616,
+     "time_stretch": 0.9961578976752661
+    }
+   }
+  },
+  {
+   "step": 10000,
+   "loss": 0.018024474542122335,
+   "val_macro_ap": 0.954420341476123,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9937778228734828,
+      "macro_ap": 0.9337738707903512,
+      "macro_f1": 0.8986676436464068
+     },
+     "segment_1s": {
+      "macro_auc": 0.992201218265692,
+      "macro_ap": 0.9327811125144017,
+      "macro_f1": 0.887803850565704
+     },
+     "clip": {
+      "macro_auc": 0.9879263209502289,
+      "macro_ap": 0.9424036580467785,
+      "macro_f1": 0.8978726557035617
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9952563867601801,
+      "macro_ap": 0.9750668121618947,
+      "macro_f1": 0.9089039881740315
+     },
+     "segment_1s": {
+      "macro_auc": 0.9937247949673709,
+      "macro_ap": 0.9721325960515701,
+      "macro_f1": 0.8937238126301982
+     },
+     "clip": {
+      "macro_auc": 0.9889874355564214,
+      "macro_ap": 0.9686517492132668,
+      "macro_f1": 0.8932338270548441
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8381335929576889,
+     "music": 0.9880661912764566,
+     "reverb": 0.9612620481757228,
+     "clipping": 0.9932908658150296,
+     "lowpass": 0.9986383146554336,
+     "highpass": 0.9924232012183175,
+     "codec": 0.5637403566830639,
+     "packet_loss": 0.9892428227227971,
+     "hum": 0.9211505027407217,
+     "clicks": 0.9724301606757375,
+     "pitch_shift": 0.9904851675787492,
+     "time_stretch": 0.996423224984497
+    },
+    "val_ru": {
+     "noise": 0.9232223810212362,
+     "music": 0.9909113537820242,
+     "reverb": 0.9931759455292286,
+     "clipping": 0.9807976161927412,
+     "lowpass": 0.9986782995711051,
+     "highpass": 0.9996343526433841,
+     "codec": 0.9074184957471816,
+     "packet_loss": 0.9858794967091588,
+     "hum": 0.9655642309317206,
+     "clicks": 0.977494330957667,
+     "pitch_shift": 0.988727822438337,
+     "time_stretch": 0.9892974204189539
+    }
+   }
+  },
+  {
+   "step": 12000,
+   "loss": 0.01652453160029836,
+   "val_macro_ap": 0.9566827395009865,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.993412718804334,
+      "macro_ap": 0.9354836100266,
+      "macro_f1": 0.8839513806527578
+     },
+     "segment_1s": {
+      "macro_auc": 0.9915742976363203,
+      "macro_ap": 0.9364013017705921,
+      "macro_f1": 0.8749545880858954
+     },
+     "clip": {
+      "macro_auc": 0.9875354467937401,
+      "macro_ap": 0.9452015990961922,
+      "macro_f1": 0.8944489799036379
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9959152864086401,
+      "macro_ap": 0.9778818689753729,
+      "macro_f1": 0.9148273504094172
+     },
+     "segment_1s": {
+      "macro_auc": 0.9937276945280051,
+      "macro_ap": 0.9741901369813677,
+      "macro_f1": 0.8981275365363289
+     },
+     "clip": {
+      "macro_auc": 0.9886806530223842,
+      "macro_ap": 0.9698573738990878,
+      "macro_f1": 0.8984071156273287
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8669984548213616,
+     "music": 0.986591216678675,
+     "reverb": 0.9638036496699833,
+     "clipping": 0.9941916935177105,
+     "lowpass": 0.9991844214091042,
+     "highpass": 0.9685641381261348,
+     "codec": 0.6020683196350568,
+     "packet_loss": 0.9934959524110555,
+     "hum": 0.9028346954337898,
+     "clicks": 0.9610046235937441,
+     "pitch_shift": 0.9909042438358977,
+     "time_stretch": 0.9961619111866852
+    },
+    "val_ru": {
+     "noise": 0.9357875462404106,
+     "music": 0.9889302271203545,
+     "reverb": 0.9937610321788829,
+     "clipping": 0.9941618806907708,
+     "lowpass": 0.9979628507357938,
+     "highpass": 0.999850335872091,
+     "codec": 0.8940165539963891,
+     "packet_loss": 0.987869359339214,
+     "hum": 0.9714447521367616,
+     "clicks": 0.9772475713040158,
+     "pitch_shift": 0.9958033184826599,
+     "time_stretch": 0.9977469996071305
+    }
+   }
+  },
+  {
+   "step": 14000,
+   "loss": 0.014825908985105343,
+   "val_macro_ap": 0.9645573355918111,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.995088705101273,
+      "macro_ap": 0.9481565958255954,
+      "macro_f1": 0.9027999309598221
+     },
+     "segment_1s": {
+      "macro_auc": 0.9937754397753776,
+      "macro_ap": 0.9486769851671145,
+      "macro_f1": 0.896573647472643
+     },
+     "clip": {
+      "macro_auc": 0.990424785350798,
+      "macro_ap": 0.9562547344317137,
+      "macro_f1": 0.912309122108398
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9966984235984825,
+      "macro_ap": 0.9809580753580267,
+      "macro_f1": 0.9334626062999197
+     },
+     "segment_1s": {
+      "macro_auc": 0.9951496784947965,
+      "macro_ap": 0.9785342009249995,
+      "macro_f1": 0.9216407850506414
+     },
+     "clip": {
+      "macro_auc": 0.9903557805728257,
+      "macro_ap": 0.9751958891866654,
+      "macro_f1": 0.9204364599907429
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8621652956810659,
+     "music": 0.9884848281655738,
+     "reverb": 0.9691527040489324,
+     "clipping": 0.9497157561324586,
+     "lowpass": 0.9991098437949472,
+     "highpass": 0.992125958668985,
+     "codec": 0.7235560919704811,
+     "packet_loss": 0.9911659131878358,
+     "hum": 0.9426075448716406,
+     "clicks": 0.9727463934537135,
+     "pitch_shift": 0.9914693351364414,
+     "time_stretch": 0.9955794847950683
+    },
+    "val_ru": {
+     "noise": 0.9448040667554358,
+     "music": 0.9871719593273258,
+     "reverb": 0.995469471639646,
+     "clipping": 0.9980642072766522,
+     "lowpass": 0.9985503078205077,
+     "highpass": 0.9998590651413983,
+     "codec": 0.9119965225218752,
+     "packet_loss": 0.988428471237224,
+     "hum": 0.9767546316234503,
+     "clicks": 0.9805183876681753,
+     "pitch_shift": 0.9941058295637946,
+     "time_stretch": 0.9957739837208368
+    }
+   }
+  },
+  {
+   "step": 16000,
+   "loss": 0.013560088443919085,
+   "val_macro_ap": 0.96864426045537,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9955276563276286,
+      "macro_ap": 0.9527747602116589,
+      "macro_f1": 0.9145941917989636
+     },
+     "segment_1s": {
+      "macro_auc": 0.9941360674039909,
+      "macro_ap": 0.9533777901009493,
+      "macro_f1": 0.9061822613395204
+     },
+     "clip": {
+      "macro_auc": 0.9918759091227026,
+      "macro_ap": 0.9603763926263187,
+      "macro_f1": 0.9205361204619789
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9970408553204062,
+      "macro_ap": 0.9845137606990813,
+      "macro_f1": 0.9418841287874797
+     },
+     "segment_1s": {
+      "macro_auc": 0.9956864804000615,
+      "macro_ap": 0.9821816585063771,
+      "macro_f1": 0.929591305809888
+     },
+     "clip": {
+      "macro_auc": 0.9914030440677801,
+      "macro_ap": 0.9793139680457633,
+      "macro_f1": 0.9277124783723186
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8565246632197835,
+     "music": 0.9908813780008551,
+     "reverb": 0.9783049167237781,
+     "clipping": 0.9829609706017582,
+     "lowpass": 0.9991838815468214,
+     "highpass": 0.9952579074447799,
+     "codec": 0.7266219236930261,
+     "packet_loss": 0.9934095947512862,
+     "hum": 0.9436263443087161,
+     "clicks": 0.9760050550464504,
+     "pitch_shift": 0.993355057182845,
+     "time_stretch": 0.9971654300198077
+    },
+    "val_ru": {
+     "noise": 0.9300053521068425,
+     "music": 0.9868889273954375,
+     "reverb": 0.9987064250432233,
+     "clipping": 0.995660225880441,
+     "lowpass": 0.9987377273645649,
+     "highpass": 0.9999045849633087,
+     "codec": 0.9607515038442969,
+     "packet_loss": 0.9896996830932128,
+     "hum": 0.9808078831198215,
+     "clicks": 0.9796541903134496,
+     "pitch_shift": 0.9964465237268623,
+     "time_stretch": 0.9969021015375155
+    }
+   }
+  },
+  {
+   "step": 18000,
+   "loss": 0.01241955547645921,
+   "val_macro_ap": 0.9653033157574205,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9955075889202204,
+      "macro_ap": 0.94626675869342,
+      "macro_f1": 0.9084376783787871
+     },
+     "segment_1s": {
+      "macro_auc": 0.9939980148548737,
+      "macro_ap": 0.9468871830032549,
+      "macro_f1": 0.9003652823595497
+     },
+     "clip": {
+      "macro_auc": 0.9911370165852181,
+      "macro_ap": 0.954151688308707,
+      "macro_f1": 0.9193570535023251
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9966789605080141,
+      "macro_ap": 0.9843398728214208,
+      "macro_f1": 0.9291065347854098
+     },
+     "segment_1s": {
+      "macro_auc": 0.995265953390137,
+      "macro_ap": 0.9823569940500058,
+      "macro_f1": 0.9192994033376888
+     },
+     "clip": {
+      "macro_auc": 0.9907893359135652,
+      "macro_ap": 0.9796623071080429,
+      "macro_f1": 0.9259425636485316
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.864288992189993,
+     "music": 0.9925594160335727,
+     "reverb": 0.9823489059684509,
+     "clipping": 0.9787468889126852,
+     "lowpass": 0.9987396962856109,
+     "highpass": 0.9683078352684484,
+     "codec": 0.6610717803702856,
+     "packet_loss": 0.9958882994793229,
+     "hum": 0.9389718290928575,
+     "clicks": 0.9800786941304804,
+     "pitch_shift": 0.9955619145192549,
+     "time_stretch": 0.9986368520700782
+    },
+    "val_ru": {
+     "noise": 0.9403487669864559,
+     "music": 0.9892050306164752,
+     "reverb": 0.9968171073702787,
+     "clipping": 0.9860252311855215,
+     "lowpass": 0.9985567634295851,
+     "highpass": 0.9998192597211872,
+     "codec": 0.9487545248573874,
+     "packet_loss": 0.9908180535180986,
+     "hum": 0.9816696851175905,
+     "clicks": 0.9835888804465721,
+     "pitch_shift": 0.9977134557011604,
+     "time_stretch": 0.9987617149067349
+    }
+   }
+  },
+  {
+   "step": 20000,
+   "loss": 0.011342740583000705,
+   "val_macro_ap": 0.9632882309454982,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9952862009215518,
+      "macro_ap": 0.9412471236374707,
+      "macro_f1": 0.9201396368740165
+     },
+     "segment_1s": {
+      "macro_auc": 0.9939294054209405,
+      "macro_ap": 0.9413706636632174,
+      "macro_f1": 0.9141079638455808
+     },
+     "clip": {
+      "macro_auc": 0.991474960879913,
+      "macro_ap": 0.9504576503453044,
+      "macro_f1": 0.9286976962278831
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9971720837575706,
+      "macro_ap": 0.9853293382535256,
+      "macro_f1": 0.9401606217227473
+     },
+     "segment_1s": {
+      "macro_auc": 0.9958852510636262,
+      "macro_ap": 0.9827327515119024,
+      "macro_f1": 0.9295312760024421
+     },
+     "clip": {
+      "macro_auc": 0.9914170741166943,
+      "macro_ap": 0.979122882602281,
+      "macro_f1": 0.9324988674330195
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8094600211885296,
+     "music": 0.9947846309118787,
+     "reverb": 0.9803034067606903,
+     "clipping": 0.9911222540834081,
+     "lowpass": 0.9991342461047975,
+     "highpass": 0.9798458636658157,
+     "codec": 0.6212755684809245,
+     "packet_loss": 0.9960884454085654,
+     "hum": 0.9505341419131699,
+     "clicks": 0.9771694506398627,
+     "pitch_shift": 0.9960411465305613,
+     "time_stretch": 0.9992063079614436
+    },
+    "val_ru": {
+     "noise": 0.9491026769688891,
+     "music": 0.9904069315751068,
+     "reverb": 0.9965361948279663,
+     "clipping": 0.9912876134759723,
+     "lowpass": 0.999225884750131,
+     "highpass": 0.9997863770856915,
+     "codec": 0.9405392090511472,
+     "packet_loss": 0.9932961535169319,
+     "hum": 0.983043035469888,
+     "clicks": 0.9849021880900684,
+     "pitch_shift": 0.9970013428983,
+     "time_stretch": 0.9988244513322132
+    }
+   }
+  },
+  {
+   "step": 22000,
+   "loss": 0.010403119354567026,
+   "val_macro_ap": 0.9613788407213695,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.995189691312147,
+      "macro_ap": 0.941561451283591,
+      "macro_f1": 0.9103291458855386
+     },
+     "segment_1s": {
+      "macro_auc": 0.9939528516224324,
+      "macro_ap": 0.9428305461151941,
+      "macro_f1": 0.9041292858657566
+     },
+     "clip": {
+      "macro_auc": 0.9919604363552449,
+      "macro_ap": 0.9525335741566469,
+      "macro_f1": 0.9227806340185611
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9966162470470571,
+      "macro_ap": 0.9811962301591479,
+      "macro_f1": 0.9212828963726855
+     },
+     "segment_1s": {
+      "macro_auc": 0.9953723083477312,
+      "macro_ap": 0.9788326657034476,
+      "macro_f1": 0.911767099700031
+     },
+     "clip": {
+      "macro_auc": 0.9907939716220698,
+      "macro_ap": 0.9758450492305134,
+      "macro_f1": 0.9183805389034325
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8035111991398619,
+     "music": 0.9940796676200904,
+     "reverb": 0.9772455321537804,
+     "clipping": 0.9770459752729225,
+     "lowpass": 0.9997717973096837,
+     "highpass": 0.9732373809530138,
+     "codec": 0.6501507613956089,
+     "packet_loss": 0.9960785538255279,
+     "hum": 0.9529375709447516,
+     "clicks": 0.9805441278409738,
+     "pitch_shift": 0.9954544032289057,
+     "time_stretch": 0.9986804457179728
+    },
+    "val_ru": {
+     "noise": 0.9502187734674352,
+     "music": 0.9922604059611878,
+     "reverb": 0.9987604247341664,
+     "clipping": 0.9988938876878212,
+     "lowpass": 0.997967071777169,
+     "highpass": 0.9998656224152197,
+     "codec": 0.8785233408575478,
+     "packet_loss": 0.9938805263543643,
+     "hum": 0.9857855274435822,
+     "clicks": 0.9828920572226381,
+     "pitch_shift": 0.9960586379109502,
+     "time_stretch": 0.9992484860776923
+    }
+   }
+  },
+  {
+   "step": 24000,
+   "loss": 0.009880208635004238,
+   "val_macro_ap": 0.9642662805647465,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9955100696965326,
+      "macro_ap": 0.9442345083401548,
+      "macro_f1": 0.9108508844583089
+     },
+     "segment_1s": {
+      "macro_auc": 0.9943897636803264,
+      "macro_ap": 0.9448498671373056,
+      "macro_f1": 0.9045204442721405
+     },
+     "clip": {
+      "macro_auc": 0.992318122385229,
+      "macro_ap": 0.9538866285633857,
+      "macro_f1": 0.9225553751506292
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9968405965704273,
+      "macro_ap": 0.9842980527893382,
+      "macro_f1": 0.9286332044998713
+     },
+     "segment_1s": {
+      "macro_auc": 0.995589241962476,
+      "macro_ap": 0.9818301252414962,
+      "macro_f1": 0.9178761737395552
+     },
+     "clip": {
+      "macro_auc": 0.9913716508347442,
+      "macro_ap": 0.9788261625076117,
+      "macro_f1": 0.9256932873206348
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8025016272494524,
+     "music": 0.9942335943096073,
+     "reverb": 0.9780883795318004,
+     "clipping": 0.9847837591569857,
+     "lowpass": 0.9995192242989398,
+     "highpass": 0.9760822857380473,
+     "codec": 0.6683934589911299,
+     "packet_loss": 0.9973130527418864,
+     "hum": 0.9535885934498749,
+     "clicks": 0.9806897896386807,
+     "pitch_shift": 0.9966637406005215,
+     "time_stretch": 0.9989565943749307
+    },
+    "val_ru": {
+     "noise": 0.9479760042870522,
+     "music": 0.992771320155561,
+     "reverb": 0.999536935228305,
+     "clipping": 0.9980788052089924,
+     "lowpass": 0.9992141255803184,
+     "highpass": 0.999801582579823,
+     "codec": 0.9137537857862241,
+     "packet_loss": 0.992718201787296,
+     "hum": 0.986821067523141,
+     "clicks": 0.9847048179562231,
+     "pitch_shift": 0.997576281722842,
+     "time_stretch": 0.9986237056562808
+    }
+   }
+  },
+  {
+   "step": 26000,
+   "loss": 0.00917209515045397,
+   "val_macro_ap": 0.9660657287768286,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.995903059225709,
+      "macro_ap": 0.9467333778446503,
+      "macro_f1": 0.910977202457994
+     },
+     "segment_1s": {
+      "macro_auc": 0.9947513719258212,
+      "macro_ap": 0.9466273421282705,
+      "macro_f1": 0.9040409165384666
+     },
+     "clip": {
+      "macro_auc": 0.9924234838070709,
+      "macro_ap": 0.9542704975449391,
+      "macro_f1": 0.922272077752606
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.997344151402523,
+      "macro_ap": 0.9853980797090068,
+      "macro_f1": 0.9340132075433923
+     },
+     "segment_1s": {
+      "macro_auc": 0.9961538710035974,
+      "macro_ap": 0.9830285646998006,
+      "macro_f1": 0.9230850131543263
+     },
+     "clip": {
+      "macro_auc": 0.9918991868692498,
+      "macro_ap": 0.9796423206393836,
+      "macro_f1": 0.9291698399833338
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8104047235770796,
+     "music": 0.9942622269238398,
+     "reverb": 0.9774150554196556,
+     "clipping": 0.9866316378523347,
+     "lowpass": 0.9994590005604352,
+     "highpass": 0.9700101360848669,
+     "codec": 0.6970676550502176,
+     "packet_loss": 0.9971967328656111,
+     "hum": 0.951225993297226,
+     "clicks": 0.9809777552730538,
+     "pitch_shift": 0.9970422719687749,
+     "time_stretch": 0.9991073452627081
+    },
+    "val_ru": {
+     "noise": 0.9573978310407696,
+     "music": 0.9935910507147944,
+     "reverb": 0.9991521949532433,
+     "clipping": 0.9968162014043537,
+     "lowpass": 0.9992919227071863,
+     "highpass": 0.9997841300227553,
+     "codec": 0.9155861608588401,
+     "packet_loss": 0.9934202896431158,
+     "hum": 0.9877361197155258,
+     "clicks": 0.9844809734490525,
+     "pitch_shift": 0.9985386128140769,
+     "time_stretch": 0.998981469184368
+    }
+   }
+  },
+  {
+   "step": 28000,
+   "loss": 0.008987142193131148,
+   "val_macro_ap": 0.9656914428086011,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9959702881582229,
+      "macro_ap": 0.9467586121337036,
+      "macro_f1": 0.9099860379647687
+     },
+     "segment_1s": {
+      "macro_auc": 0.9948539317965568,
+      "macro_ap": 0.9467346350046789,
+      "macro_f1": 0.9023104744907596
+     },
+     "clip": {
+      "macro_auc": 0.9925031114545173,
+      "macro_ap": 0.954720746409941,
+      "macro_f1": 0.9201865903743557
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.9972350457008915,
+      "macro_ap": 0.9846242734834986,
+      "macro_f1": 0.9329749954675988
+     },
+     "segment_1s": {
+      "macro_auc": 0.9960075454203697,
+      "macro_ap": 0.9819603801296602,
+      "macro_f1": 0.9225104485563435
+     },
+     "clip": {
+      "macro_auc": 0.9916871126936554,
+      "macro_ap": 0.9783272581817896,
+      "macro_f1": 0.9280379600949026
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.822091753276001,
+     "music": 0.9939003395284468,
+     "reverb": 0.9779179037607133,
+     "clipping": 0.9857314646860196,
+     "lowpass": 0.999593384479317,
+     "highpass": 0.9806052995699851,
+     "codec": 0.6769525382725714,
+     "packet_loss": 0.9971441541765772,
+     "hum": 0.9503837811249735,
+     "clicks": 0.9809077294479177,
+     "pitch_shift": 0.9968377554361978,
+     "time_stretch": 0.9990372418457217
+    },
+    "val_ru": {
+     "noise": 0.9562612989350948,
+     "music": 0.9931557805590477,
+     "reverb": 0.9990852472094756,
+     "clipping": 0.9982918764215263,
+     "lowpass": 0.9993638811447602,
+     "highpass": 0.9998304263815657,
+     "codec": 0.9066800347092814,
+     "packet_loss": 0.9933922169146713,
+     "hum": 0.9877707907635018,
+     "clicks": 0.9841204890223741,
+     "pitch_shift": 0.998555099182636,
+     "time_stretch": 0.9989841405580471
+    }
+   }
+  },
+  {
+   "step": 30000,
+   "loss": 0.008822277313505765,
+   "val_macro_ap": 0.9657940950608517,
+   "sets": {
+    "val_en": {
+     "frame": {
+      "macro_auc": 0.9959758493137625,
+      "macro_ap": 0.9469142650178455,
+      "macro_f1": 0.9094282643671651
+     },
+     "segment_1s": {
+      "macro_auc": 0.9948591906189098,
+      "macro_ap": 0.947049250683202,
+      "macro_f1": 0.9018388567624277
+     },
+     "clip": {
+      "macro_auc": 0.9924404873031835,
+      "macro_ap": 0.9546272328996436,
+      "macro_f1": 0.920216583516592
+     }
+    },
+    "val_ru": {
+     "frame": {
+      "macro_auc": 0.997280079523683,
+      "macro_ap": 0.9846739251038579,
+      "macro_f1": 0.9339026434272615
+     },
+     "segment_1s": {
+      "macro_auc": 0.9960374063703222,
+      "macro_ap": 0.9819967535241277,
+      "macro_f1": 0.9237394352822492
+     },
+     "clip": {
+      "macro_auc": 0.9917283128775577,
+      "macro_ap": 0.9783761808498371,
+      "macro_f1": 0.9281659692284285
+     }
+    }
+   },
+   "per_class_ap": {
+    "val_en": {
+     "noise": 0.8232775877067415,
+     "music": 0.9937834785945643,
+     "reverb": 0.9788712780170233,
+     "clipping": 0.9839683519553726,
+     "lowpass": 0.9995998716256836,
+     "highpass": 0.978254301156749,
+     "codec": 0.6818443063541599,
+     "packet_loss": 0.9972538963871094,
+     "hum": 0.9491641751993553,
+     "clicks": 0.9810993053060276,
+     "pitch_shift": 0.9967966397505934,
+     "time_stretch": 0.9990579881607662
+    },
+    "val_ru": {
+     "noise": 0.9566068954463757,
+     "music": 0.9934421733936598,
+     "reverb": 0.9993168728011599,
+     "clipping": 0.9982998046790912,
+     "lowpass": 0.9993197204568725,
+     "highpass": 0.9998259782450261,
+     "codec": 0.9070000318755437,
+     "packet_loss": 0.9934219011585418,
+     "hum": 0.987176792358621,
+     "clicks": 0.9841722315202732,
+     "pitch_shift": 0.9985523765879143,
+     "time_stretch": 0.9989523227232169
+    }
+   }
+  }
+ ],
+ "best": 0.96864426045537
+}