---
license: other
license_name: apache-2.0-and-mit
license_link: https://huggingface.co/DeafBoyAudio/audio-compass-models/blob/main/LICENSE-CLAP
tags:
  - audio
  - onnx
  - clap
  - crepe
  - pitch-detection
  - zero-shot-audio-classification
library_name: onnx
pipeline_tag: audio-classification
---

# Audio Compass models

ONNX models used by **Audio Compass**, a sample-finder plugin by Deaf Boy Audio. The plugin downloads
these files itself (*⋯ › Download models*) and checks every file against `SHA256SUMS`, so there is
nothing to do by hand.

| File | What it is | Licence |
|---|---|---|
| `clap_audio.onnx` | CLAP audio encoder: 48 kHz mono, 10 s → 512-d L2-normalised embedding. Log-mel front end built into the graph. Dynamic int8 weights (front end kept fp32). | Apache-2.0 |
| `clap_text.onnx` | CLAP text encoder: token ids → 512-d L2-normalised embedding. Dynamic int8. | Apache-2.0 |
| `clap_vocab.json`, `clap_merges.txt` | RoBERTa byte-level BPE tokenizer files | Apache-2.0 |
| `clap_config.json` | Every preprocessing parameter (sample rate, crop/pad rule, mel settings, token ids) | Apache-2.0 |
| `crepe_tiny.onnx` | CREPE "tiny" pitch tracker: 1024-sample frames @ 16 kHz → 360-bin salience. Per-frame normalisation built into the graph. | MIT |
| `SHA256SUMS` | Checksums of the files above | — |

## Where they come from (modified works)

- **CLAP**: converted from [`laion/larger_clap_music_and_speech`](https://huggingface.co/laion/larger_clap_music_and_speech)
  (LAION, Apache-2.0). Changes: exported to ONNX as separate audio and text encoders with projection +
  L2 normalisation, mel spectrogram moved into the graph, weights dynamically quantised to int8.
  Licence: [LICENSE-CLAP](LICENSE-CLAP).
- **CREPE tiny**: the weights distributed with [`torchcrepe`](https://github.com/maxrmorrison/torchcrepe) 0.0.24
  (MIT, © 2020 Max Morrison), a port of [CREPE](https://github.com/marl/crepe) (MIT, © 2018 Jong Wook Kim,
  Justin Salamon, Peter Li, Juan Pablo Bello). Changes: exported to ONNX with per-frame normalisation in the
  graph. Licence: [LICENSE-CREPE](LICENSE-CREPE).

The files are produced by `tools/export_models.py` in the Audio Compass source code; its parity tests
check the ONNX outputs against the original PyTorch models (cosine ≥ 0.99 for CLAP, within 10 cents for CREPE).

## Citations

- Wu, Chen, Zhang, Hui, Berg-Kirkpatrick, Dubnov. *Large-scale Contrastive Language-Audio Pretraining with
  Feature Fusion and Keyword-to-Caption Augmentation*. ICASSP 2023.
- Kim, Salamon, Li, Bello. *CREPE: A Convolutional Representation for Pitch Estimation*. ICASSP 2018.