|
Download README.md from thelamapi/Next-Echo: direct link, hf CLI and curl.
- Browser
- Download file 3.62 kB
-
https://huggingface.co/spaces/thelamapi/Next-Echo/resolve/main/README.md
- Command line
-
hf download hf://spaces/thelamapi/Next-Echo/README.md
-
curl -L -o README.md https://huggingface.co/spaces/thelamapi/Next-Echo/resolve/main/README.md
3.62 kB
| title: Next Echo | |
| emoji: π | |
| colorFrom: green | |
| colorTo: gray | |
| sdk: static | |
| app_file: index.html | |
| pinned: false | |
| license: cc-by-nc-3.0 | |
| short_description: Find an example sound inside a recording, live with your mic | |
| # Next Echo | |
| Next Echo finds **every moment a given example sound occurs** in a longer recording β a clap, a knock, a bark β with no | |
| fixed class list. A small CNN embeds 1-second log-mel windows; the recording is scanned against the example. | |
| It is compared with plain log-mel similarity; the final score fuses both after per-recording normalisation. | |
| **Model:** https://huggingface.co/thelamapi/Next-Echo β record a 2-second example with your microphone (or pick one), choose a recording, and listen to each hit. Runs in the browser. | |
| ## Results on constructed 30 s test recordings (ESC-50 fold 5; 40 recordings per cell) | |
| Event F1 (Β±0.5 s) / false alarms per minute β log-mel baseline vs Next Echo (better F1 in bold): | |
| | Classes / condition | Log-mel similarity | Next Echo (learned) | | |
| |---|---:|---:| | |
| | seen/clean | **0.676** / 2.15 | 0.619 / 3.25 | | |
| | seen/noisy | 0.606 / 1.45 | **0.630** / 2.30 | | |
| | seen/overlap | **0.788** / 0.80 | 0.749 / 1.05 | | |
| | seen/absent | β / 4.00 | β / 4.95 | | |
| | unseen/clean | 0.617 / 2.10 | **0.658** / 3.30 | | |
| | unseen/noisy | 0.667 / 1.10 | **0.717** / 2.05 | | |
| | unseen/overlap | **0.790** / 0.85 | 0.767 / 1.70 | | |
| | unseen/absent | β / 3.65 | β / 5.60 | | |
| | seen/all-present | **0.690** / 1.47 | 0.662 / 2.20 | | |
| | unseen/all-present | 0.691 / 1.35 | **0.712** / 2.35 | | |
| *unseen* = the 10 classes never used in training (clapping, door_wood_knock, rooster, church_bells, glass_breaking, keyboard_typing, siren, cow, sneezing, can_opening). *absent* recordings contain no target, so | |
| only false alarms count. Thresholds were chosen on validation recordings built from fold 4. | |
|  | |
| ## Method | |
| * Audio: 16 kHz mono, 512-point STFT (hop 256), 64 mel bands (50β8000 Hz), log. | |
| * Embedding: 4 conv blocks + linear β 64-d, supervised contrastive loss (Ο = 0.1) on random 1-s crops of 960 clips from 40 classes, with gain, mixing and noise augmentation, 5000 steps. | |
| * Training windows mimic the test conditions: real waveform segments mixed with another sound and brown noise (0β30 dB SNR), alongside clean segments like the user's example. | |
| * Query: the loudest 1-s windows of the example are embedded and averaged. Recording: windows every 0.256 s. | |
| * Score: per-recording robust normalisation (median/MAD), then Next Echo = Ξ±Β·z(learned) + (1βΞ±)Β·z(log-mel) with Ξ± = 0.75 chosen on validation recordings | |
| (options Ξ± β {1, 0.75, 0.5}). Consecutive windows above the threshold form one detection. | |
| * Baseline: cosine between per-band mean/std of log-mel windows, raw or normalised β whichever validated better (raw). | |
| * The browser computes the same log-mel (own FFT) and CNN with `nlrt.js`; Node parity on 3 real snippets: max abs difference 7.105427357601002e-15. | |
| ## Usage | |
| ```bash | |
| python inference.py --query clap.wav --recording scene.wav # 16 kHz mono 16-bit WAV | |
| python inference.py --selftest | |
| ``` | |
| ## Limitations and license | |
| * Recordings are mixtures of real clips, not naturally recorded scenes; real rooms add reverberation and far more varied backgrounds. | |
| * Very short or very quiet examples give weak queries. Sounds that resemble each other (knock vs footsteps) cause false alarms. | |
| * **Non-commercial:** ESC-50 is CC BY-NC 3.0, so this model, its demo audio and the Space are for non-commercial use only. | |
| * Trained on NVIDIA L4; profile full. Failure cases are listed in `evaluation.json`. | |