Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
pipeline_tag: audio-classification
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Model Card for DualCounter
|
| 7 |
+
|
| 8 |
+
Class-agnostic audio repetition counting: counting repeated sounds in an audio waveform (a clock striking, hammer blows, a heartbeat, etc.) without training on those specific sound classes.
|
| 9 |
+
|
| 10 |
+
Code: https://github.com/Hamozwa/audio-counting
|
| 11 |
+
|
| 12 |
+
Paper: [link] 路 Demo: https://huggingface.co/spaces/Hamozwa/dualcounter
|
| 13 |
+
|
| 14 |
+
Datasets: https://huggingface.co/datasets/Hamozwa/RepeatSynth 路 https://huggingface.co/datasets/Hamozwa/RepeatReal
|
| 15 |
+
|
| 16 |
+
## Model Description
|
| 17 |
+
|
| 18 |
+
DualCounter runs two independent counting architectures and combines their predictions:
|
| 19 |
+
|
| 20 |
+
- **WavCounter** regresses a repetition heatmap over the waveform and derives a count from it via a Schmitt trigger.
|
| 21 |
+
- **TSSMCounter** builds a temporal self-similarity matrix from Wav2Vec2 features and classifies the count directly with a DINO vision transformer.
|
| 22 |
+
|
| 23 |
+
DualCounter runs both and falls back to TSSMCounter's prediction when the WavCounter heatmap is judged low quality.
|
| 24 |
+
|
| 25 |
+
## Repository Contents
|
| 26 |
+
|
| 27 |
+
```
|
| 28 |
+
wav_counter/ checkpoint for the WavCounter model
|
| 29 |
+
tssm_counter/ checkpoint for the TSSMCounter model
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
## Usage
|
| 33 |
+
|
| 34 |
+
Load and run these checkpoints using the training/inference code in the linked GitHub repo (`WavCounter_tester.py`, `TSSMCounter_tester.py`, `DualCounter.py`). DINO weights are pulled automatically via `torch.hub` (`facebookresearch/dino`) when running TSSMCounter.
|
| 35 |
+
|
| 36 |
+
## Training Data
|
| 37 |
+
|
| 38 |
+
Trained on the synthetic datasets in RepeatSynth. Evaluated zero-shot on the real-world recordings in RepeatReal, covering mechanical, medical, and ecological domains.
|
| 39 |
+
|
| 40 |
+
## Citation
|
| 41 |
+
|
| 42 |
+
```bibtex
|
| 43 |
+
[Your paper's BibTeX entry here]
|
| 44 |
+
```
|