| --- |
| license: apache-2.0 |
| library_name: pytorch |
| pipeline_tag: audio-classification |
| base_model: |
| - justinchuby/Perch-onnx |
| - wrice/perch-v2-efficientnet-b3 |
| tags: |
| - audio |
| - bioacoustics |
| - bird-classification |
| - das |
| - embeddings |
| --- |
| |
| # PERCH 2 PyTorch |
|
|
| PyTorch implementation of the complete PERCH 2 waveform model for |
| bioacoustic classification and 1536-dimensional audio embeddings. |
|
|
| ## Load From Hugging Face |
|
|
| ```python |
| import torch |
| |
| model = torch.hub.load("janclemenslab/perch2_torch", "perch_v2").eval() |
| waveform = torch.zeros(5 * 32_000) # Mono 32 kHz audio. |
| |
| with torch.no_grad(): |
| outputs = model(waveform) |
| |
| index = outputs["label"][0].argmax() |
| print(model.labels[index]) |
| ``` |
|
|
| This call downloads `perch_v2_torch.pt` from this Hugging Face repository and |
| caches it locally. |
|
|
| Inputs are mono, 32 kHz waveforms with shape `(time,)` or `(batch, time)`. |
| Audio longer than five seconds is processed in overlapping windows and pooled. |
|
|
| ## Outputs |
|
|
| - `embedding`: `(batch, 1536)` global embedding |
| - `spatial_embedding`: `(batch, time, frequency, 1536)` unpooled embedding |
| - `spectrogram`: log-mel spectrogram |
| - `label`: `(batch, 14795)` uncalibrated class logits |
|
|
| `model.labels[index]` maps a logit index to its embedded class name. Thresholds |
| should be calibrated for the target data. |
|
|
| ## Reproducibility |
|
|
| The conversion code, demo notebook, and verification command are available in |
| [janclemenslab/perch2_torch](https://github.com/janclemenslab/perch2_torch). |
|
|
| ## Attribution And License |
|
|
| Derived from the Apache-2.0 [PERCH ONNX model](https://huggingface.co/justinchuby/Perch-onnx) |
| and Apache-2.0 [wrice EfficientNet-B3 checkpoint](https://huggingface.co/wrice/perch-v2-efficientnet-b3). |
| The class taxonomy comes from the Apache-2.0 [PERCH model release](https://huggingface.co/cgeorgiaw/Perch). |
|
|
