Configuration Parsing Warning:In adapter_config.json: "peft.task_type" must be a string
Crate — CLAP LoRA for producer vocabulary
LoRA adapter over laion/clap-htsat-unfused fine-tuned so producer terms base CLAP
barely knows — boom-bap, tape-saturated, rimshot, reese bass — pull the right
audio. Part of Crate, sound-native
search for music producers (hum / drop a track / describe it → one embedding space).
Results — producer-vocab retrieval (held-out)
| metric | base CLAP | fine-tuned |
|---|---|---|
| recall@1 | 0.210 | 0.405 |
| recall@10 | 0.746 | 0.951 |
| recall@5 | 0.580 | 0.868 |
Usage
from transformers import ClapModel, ClapProcessor
from peft import PeftModel
base = "laion/clap-htsat-unfused"
model = PeftModel.from_pretrained(ClapModel.from_pretrained(base), "jahnaviym/crate-clap-lora").merge_and_unload()
proc = ClapProcessor.from_pretrained(base)
# proc(text=[...]) / proc(audios=[...], sampling_rate=48000) → get_text/audio_features
Training
- Symmetric InfoNCE on (audio, text) pairs from Freesound (CC0/CC-BY) + free packs.
- LoRA on the audio/text projection heads (r=16, alpha=32).
- Label-preserving augmentation (pitch/stretch/noise/EQ) for positive pairs.
See the repo for the full pipeline and eval.
- Downloads last month
- -
Model tree for jahnaviym/crate-clap-lora
Base model
laion/clap-htsat-unfused