eval/README.md

Evaluation

The scripts behind the numbers on the model card. You only need uv.

uv sync
uv run song_describer.py jaisawhney/SonicEmbed
uv run clap_eval.py
uv run mteb_tasks.py jaisawhney/SonicEmbed --output-dir mteb-results/sonicembed

Models can be a Hugging Face ID or a local folder, so google/embeddinggemma-2 scores the base model. Add --device mps on Apple silicon or --device cpu if you don't have an NVIDIA GPU.

Song Describer downloads 3.3 GB on the first run.