--- license: mit tags: - audio - sound-event-detection - noise-detection - vaani - indoml-2026 datasets: - ARTPARK-IISc/Vaani-Noise-Event-Dataset base_model: - openai/whisper-large-v3 - openai/whisper-large-v2 - openai/whisper-medium - openai/whisper-small - microsoft/wavlm-base-plus --- # CodeAMU — IndoML 2026 Datathon, Track 1 (Noise Event Detection): detector checkpoints The 24 frame-level noise-event detectors of team **CodeAMU**'s Track 1 system, **1st place on the final (Phase 2) leaderboard: Combined 1.6173** (Event F1 0.7895 + Segment Dice 0.8278). Code, inference pipeline, training configurations and technical report: (`python scripts/download_checkpoints.py` there downloads this repository into `checkpoints/`). Authors: Shah Ahmed Shakir Abu Asim Khan (Aligarh Muslim University), Sajid Javid (IIIT Delhi), Osama Zaheer (Binary Semantics), Ahraz Shamim (Vecmocon Technologies). Contact: sajidj@iiitd.ac.in ## Contents `detectors//best.pt` — PyTorch checkpoint (`model` state dict, training `args`, `decode_params`, `frame_sec`); `detectors//best.json` — validation score of the checkpoint. Load with `inference/infer.py` of the GitHub repository (it rebuilds the architecture from `args`; `hr_*` detectors need `SED_TIME_POOL=1`). Every detector outputs per-frame noise presence, onset and offset probabilities (plus 7 category and clip-level tag heads) on a 20 ms grid (10 ms for `hr_*`). The final system routes each clip to a natural or a synthetic ensemble of these detectors and post-processes the result (see the report). | Detector | Encoder | Frame | Training data | Initialised from | Scored on | Combined | |---|---|---|---|---|---|---| | `beats_full_gold` | BEATs + Whisper medium | 20 ms | all gold + all validation (13,612) | rb_beats_gold | official validation (2,501) | 1.5984 ¹ | | `bg_lg3_syn_gold` | Whisper large-v3 | 20 ms | all 11,111 gold clips | bg_lg3_syn | official validation (2,501) | 1.4186 | | `bg_lg3_syn` | Whisper large-v3 | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4242 | | `bg_sml_syn` | Whisper small | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4022 | | `cb_big` | CRNN (128-mel) | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4136 | | `cb_s7_gold` | CRNN (128-mel) | 20 ms | all 11,111 gold clips | cb_s7 | official validation (2,501) | 1.4128 | | `cb_s7` | CRNN (128-mel) | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4076 | | `gv_lg3_A` | Whisper large-v3 | 20 ms | gold + validation half B | bg_lg3_syn | validation half A (1,259) | 1.4371 | | `gv_lg3_B` | Whisper large-v3 | 20 ms | gold + validation half A | bg_lg3_syn | validation half B (1,242) | 1.4468 | | `gv_med_A` | Whisper medium | 20 ms | gold + validation half B | ov_med_syn | validation half A (1,259) | 1.4369 | | `gv_med_B` | Whisper medium | 20 ms | gold + validation half A | ov_med_syn | validation half B (1,242) | 1.4351 | | `gv_sml_A` | Whisper small | 20 ms | gold + validation half B | bg_sml_syn | validation half A (1,259) | 1.4220 | | `gv_sml_B` | Whisper small | 20 ms | gold + validation half A | bg_sml_syn | validation half B (1,242) | 1.4210 | | `hr_med_syn_gold` | Whisper medium | 10 ms | all 11,111 gold clips | hr_med_syn | official validation (2,501) | 1.3810 | | `hr_med_syn` | Whisper medium | 10 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.3973 | | `lg2_full_gold` | Whisper large-v2 | 20 ms | all gold + all validation (13,612) | v9_lg2_gold | official validation (2,501) | 1.6190 ¹ | | `lg3_full_gold` | Whisper large-v3 | 20 ms | all gold + all validation (13,612) | v9_lg3_s21 | official validation (2,501) | 1.6547 ¹ | | `ov_med_syn` | Whisper medium | 20 ms | all tiers + 11,762 synthetic clips | — | official validation (2,501) | 1.4062 | | `rb_beats_gold` | BEATs + Whisper medium | 20 ms | 7,112 gold clips (3,999 held out) | rb_beats | gold hold-out (3,999) | 1.0840 | | `v2_crnn_gold` | CRNN (64-mel) | 20 ms | 10,000 gold clips (hold-out excluded) | v2_crnn | gold hold-out (1,111) | 1.0697 | | `v4_wavlm_gold` | WavLM Base+ | 20 ms | 10,000 gold clips (hold-out excluded) | v4_wavlm | gold hold-out (1,111) | 1.0607 | | `v8_whis_lg_gold` | Whisper large-v3 | 20 ms | 10,000 gold clips (hold-out excluded) | v8_whis_lg | gold hold-out (1,111) | 1.1102 | | `v9_lg2_gold` | Whisper large-v2 | 20 ms | 10,000 gold clips (hold-out excluded) | v9_lg2 | gold hold-out (1,111) | 1.1037 | | `v9_lg3_s21` | Whisper large-v3 | 20 ms | all tiers minus a 1,111-clip gold hold-out | — | gold hold-out (1,111) | 1.1119 | ¹ in-sample: the validation clips are part of the training data of the `*_full_gold` models. Scores on different sets are not comparable (the gold hold-outs contain natural clips only and are harder). ## Pretrained encoders | Encoder | Source (revision) | |---|---| | Whisper small / medium / large-v2 / large-v3 | `openai/whisper-*` (973afd2 / abdf7c3 / ae46427 / 06f233f) | | WavLM Base+ | `microsoft/wavlm-base-plus` (4c66d48) | | BEATs iter3+ AS2M (cpt2), frozen | `WeiChihChen/BEATs_iter3_plus_AS2M_finetuned_on_AS2M_cpt2` (172e407), mirror of Microsoft UniLM BEATs | ## Training data `ARTPARK-IISc/Vaani-Noise-Event-Dataset` (90,635 clips; gold / silver / bronze annotation tiers), synthetic clips built from its gold tier, and the official IndoML 2026 Track 1 validation release (2,501 clips).