File size: 1,875 Bytes
8357563 c536169 8357563 95c5da3 8357563 95c5da3 8357563 c536169 8357563 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 | ---
license: mit
tags:
- music
- music-evaluation
- musicgen
- audio
library_name: pytorch
---
# MEva evaluator checkpoints
Trained CNN evaluators for the ISMIR 2026 paper
**"Do Music Generative Models Understand Musical Qualities? Automatic Music Evaluation with Model-Intrinsic Signals"**
(Xiaosha Li, Chun Liu, Ziyu Wang).
- Code: https://github.com/YoEv/MEva
- Demo: https://yoev.github.io/MEva
Each checkpoint maps intrinsic signals of a frozen MusicGen (per-token loss,
predictive entropy, and SAE latents under teacher forcing, delay pattern) to a
human quality rating. 14 variants: 7 signal combinations × MusicGen-{small,large}.
## Layout
```
per_benchmark/{small,large}/{musiceval,songeval,aime,musicpref,music_arena}/
f01_loss_only_cnn_clean_best.pth
f02_entropy_only_cnn_clean_best.pth
f03_sae_only_cnn_clean_best.pth
f04_loss_entropy_cnn_clean_best.pth
f05_entropy_sae_cnn_clean_best.pth
f06_loss_sae_cnn_clean_best.pth
f07_loss_entropy_sae_cnn_clean_best.pth
pooled/{small,large}/ # trained on the union of the five benchmarks
(same seven files)
```
`f01`–`f07` follow the paper's experiment registry (single-signal, pairwise,
and full hybrids). Per-benchmark models produce Table 1's per-benchmark
columns; `pooled/` models produce the *All benchmarks* column.
## Usage
See the [training/eval scripts](https://github.com/YoEv/MEva) —
`training/hybrid/train_cnn.py` (architecture) and
`scripts/eval/eval_14_experiments.py` (evaluation protocol).
## Citation
```bibtex
@inproceedings{li2026meva,
title = {Do Music Generative Models Understand Musical Qualities? Automatic Music Evaluation with Model-Intrinsic Signals},
author = {Li, Xiaosha and Liu, Chun and Wang, Ziyu},
booktitle = {Proc. of the 27th Int. Society for Music Information Retrieval Conf. (ISMIR)},
year = {2026}
}
```
|