| --- |
| license: mit |
| tags: |
| - audio |
| - sound-event-detection |
| - noise-detection |
| - vaani |
| - indoml-2026 |
| datasets: |
| - ARTPARK-IISc/Vaani-Noise-Event-Dataset |
| base_model: |
| - openai/whisper-large-v3 |
| - openai/whisper-large-v2 |
| - openai/whisper-medium |
| - openai/whisper-small |
| - microsoft/wavlm-base-plus |
| --- |
| |
| # CodeAMU — IndoML 2026 Datathon, Track 1 (Noise Event Detection): detector checkpoints |
|
|
| The 24 frame-level noise-event detectors of team **CodeAMU**'s Track 1 system, **1st place on the final (Phase 2) |
| leaderboard: Combined 1.6173** (Event F1 0.7895 + Segment Dice 0.8278). |
|
|
| Code, inference pipeline, training configurations and technical report: <https://github.com/sajidjavid222/CodeAMU_Track1> |
| (`python scripts/download_checkpoints.py` there downloads this repository into `checkpoints/`). |
|
|
| Authors: Shah Ahmed Shakir Abu Asim Khan (Aligarh Muslim University), Sajid Javid (IIIT Delhi), Osama Zaheer |
| (Binary Semantics), Ahraz Shamim (Vecmocon Technologies). Contact: sajidj@iiitd.ac.in |
|
|
| ## Contents |
|
|
| `detectors/<name>/best.pt` — PyTorch checkpoint (`model` state dict, training `args`, `decode_params`, `frame_sec`); |
| `detectors/<name>/best.json` — validation score of the checkpoint. Load with `inference/infer.py` of the GitHub |
| repository (it rebuilds the architecture from `args`; `hr_*` detectors need `SED_TIME_POOL=1`). |
|
|
| Every detector outputs per-frame noise presence, onset and offset probabilities (plus 7 category and clip-level tag |
| heads) on a 20 ms grid (10 ms for `hr_*`). The final system routes each clip to a natural or a synthetic ensemble of |
| these detectors and post-processes the result (see the report). |
|
|
| | Detector | Encoder | Frame | Training data | Initialised from | Scored on | Combined | |
| |---|---|---|---|---|---|---| |
| | `beats_full_gold` | BEATs + Whisper medium | 20 ms | all gold + all validation (13,612) | rb_beats_gold | official validation (2,501) | 1.5984 ¹ | |
| | `bg_lg3_syn_gold` | Whisper large-v3 | 20 ms | all 11,111 gold clips | bg_lg3_syn | official validation (2,501) | 1.4186 | |
| | `bg_lg3_syn` | Whisper large-v3 | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4242 | |
| | `bg_sml_syn` | Whisper small | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4022 | |
| | `cb_big` | CRNN (128-mel) | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4136 | |
| | `cb_s7_gold` | CRNN (128-mel) | 20 ms | all 11,111 gold clips | cb_s7 | official validation (2,501) | 1.4128 | |
| | `cb_s7` | CRNN (128-mel) | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4076 | |
| | `gv_lg3_A` | Whisper large-v3 | 20 ms | gold + validation half B | bg_lg3_syn | validation half A (1,259) | 1.4371 | |
| | `gv_lg3_B` | Whisper large-v3 | 20 ms | gold + validation half A | bg_lg3_syn | validation half B (1,242) | 1.4468 | |
| | `gv_med_A` | Whisper medium | 20 ms | gold + validation half B | ov_med_syn | validation half A (1,259) | 1.4369 | |
| | `gv_med_B` | Whisper medium | 20 ms | gold + validation half A | ov_med_syn | validation half B (1,242) | 1.4351 | |
| | `gv_sml_A` | Whisper small | 20 ms | gold + validation half B | bg_sml_syn | validation half A (1,259) | 1.4220 | |
| | `gv_sml_B` | Whisper small | 20 ms | gold + validation half A | bg_sml_syn | validation half B (1,242) | 1.4210 | |
| | `hr_med_syn_gold` | Whisper medium | 10 ms | all 11,111 gold clips | hr_med_syn | official validation (2,501) | 1.3810 | |
| | `hr_med_syn` | Whisper medium | 10 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.3973 | |
| | `lg2_full_gold` | Whisper large-v2 | 20 ms | all gold + all validation (13,612) | v9_lg2_gold | official validation (2,501) | 1.6190 ¹ | |
| | `lg3_full_gold` | Whisper large-v3 | 20 ms | all gold + all validation (13,612) | v9_lg3_s21 | official validation (2,501) | 1.6547 ¹ | |
| | `ov_med_syn` | Whisper medium | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4062 | |
| | `rb_beats_gold` | BEATs + Whisper medium | 20 ms | 7,112 gold clips (3,999 held out) | rb_beats | gold hold-out (3,999) | 1.0840 | |
| | `v2_crnn_gold` | CRNN (64-mel) | 20 ms | 10,000 gold clips (hold-out excluded) | v2_crnn | gold hold-out (1,111) | 1.0697 | |
| | `v4_wavlm_gold` | WavLM Base+ | 20 ms | 10,000 gold clips (hold-out excluded) | v4_wavlm | gold hold-out (1,111) | 1.0607 | |
| | `v8_whis_lg_gold` | Whisper large-v3 | 20 ms | 10,000 gold clips (hold-out excluded) | v8_whis_lg | gold hold-out (1,111) | 1.1102 | |
| | `v9_lg2_gold` | Whisper large-v2 | 20 ms | 10,000 gold clips (hold-out excluded) | v9_lg2 | gold hold-out (1,111) | 1.1037 | |
| | `v9_lg3_s21` | Whisper large-v3 | 20 ms | all tiers minus a 1,111-clip gold hold-out | — | gold hold-out (1,111) | 1.1119 | |
| |
| ¹ in-sample: the validation clips are part of the training data of the `*_full_gold` models. |
| Scores on different sets are not comparable (the gold hold-outs contain natural clips only and are harder). |
| |
| ## Pretrained encoders |
| |
| | Encoder | Source (revision) | |
| |---|---| |
| | Whisper small / medium / large-v2 / large-v3 | `openai/whisper-*` (973afd2 / abdf7c3 / ae46427 / 06f233f) | |
| | WavLM Base+ | `microsoft/wavlm-base-plus` (4c66d48) | |
| | BEATs iter3+ AS2M (cpt2), frozen | `WeiChihChen/BEATs_iter3_plus_AS2M_finetuned_on_AS2M_cpt2` (172e407), mirror of Microsoft UniLM BEATs | |
|
|
| ## Training data |
|
|
| `ARTPARK-IISc/Vaani-Noise-Event-Dataset` (90,635 clips; gold / silver / bronze annotation tiers), synthetic clips |
| built from its gold tier, and the official IndoML 2026 Track 1 validation release (2,501 clips). |
|
|
