Classifier heads for BEATs and AST encoders
Browse files
README.md
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: mixed-see-card
|
| 4 |
+
library_name: meow-decoder
|
| 5 |
+
pipeline_tag: audio-classification
|
| 6 |
+
tags: [cats, bioacoustics, meow, audio-classification]
|
| 7 |
+
datasets: [CatMeows]
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
# Meow Decoder classifier heads
|
| 11 |
+
|
| 12 |
+
Small logistic-regression heads that turn a meow's audio embedding into the likeliest situation:
|
| 13 |
+
waiting for food, enjoying attention, or stressed somewhere new. Used by
|
| 14 |
+
[meow-decoder](https://github.com/ATHARV-IIT-BOMBAY/meow-decoder), which downloads them automatically.
|
| 15 |
+
|
| 16 |
+
| File | Encoder | First guess right (unseen cats) | Top two | Calibration error | License |
|
| 17 |
+
|---|---|---|---|---|---|
|
| 18 |
+
| `beats.npz` | Earth Species Project `esp_aves2_sl_beats_all` | 65.9% | 89.1% | 2.2% | CC BY-NC-SA 4.0 (follows the encoder) |
|
| 19 |
+
| `ast.npz` | MIT AST fine-tuned on AudioSet | 61.8% | 84.1% | 4.7% | CC BY 4.0 (follows the CatMeows data) |
|
| 20 |
+
|
| 21 |
+
Always guessing the most common situation scores 50.2%. Scores are 5-fold cross-validation over whole cats
|
| 22 |
+
(`splits.json` in the code repo), so every test meow comes from a cat the head never saw.
|
| 23 |
+
|
| 24 |
+
## Format
|
| 25 |
+
|
| 26 |
+
Plain NumPy arrays, no pickle: `classes`, `mean`, `scale`, `coef`, `intercept` (standardize, then multinomial
|
| 27 |
+
logistic regression), plus `X`, `y` (the 440 training embeddings, kept so the app can refit with your own cat's meows).
|
| 28 |
+
|
| 29 |
+
```python
|
| 30 |
+
from meow_decoder import Decoder
|
| 31 |
+
Decoder("beats").predict("meow.wav")["results"]
|
| 32 |
+
```
|
| 33 |
+
|
| 34 |
+
## Training data
|
| 35 |
+
|
| 36 |
+
[CatMeows](https://zenodo.org/records/4008297) (Ludovico et al., 2020): 440 meows from 21 Maine Coon and European
|
| 37 |
+
Shorthair cats, CC BY 4.0. Audio is 8 kHz, so nothing above 4 kHz was heard.
|
| 38 |
+
|
| 39 |
+
## Limits
|
| 40 |
+
|
| 41 |
+
Three situations only. It cannot detect pain or illness. Pet cats from one dataset; other cats may sound different.
|
ast.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0978d3ae2960cf914cfaf3bb8bc85f997a0e26fd20dd5d9e68782ebcdc0338ab
|
| 3 |
+
size 1266514
|
beats.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4c247a5b586a0f2c730a4ee8bd7ad60683c2e373ea4bd0cb65c845c50edf6d1c
|
| 3 |
+
size 1272342
|