---
license: mit
tags:
- music
- music-evaluation
- musicgen
- audio
library_name: pytorch
---

# MEva evaluator checkpoints

Trained CNN evaluators for the ISMIR 2026 paper
**"Do Music Generative Models Understand Musical Qualities? Automatic Music Evaluation with Model-Intrinsic Signals"**
(Xiaosha Li, Chun Liu, Ziyu Wang).

- Code: https://github.com/YoEv/MEva
- Demo: https://yoev.github.io/MEva

Each checkpoint maps intrinsic signals of a frozen MusicGen (per-token loss,
predictive entropy, and SAE latents under teacher forcing, delay pattern) to a
human quality rating. 14 variants: 7 signal combinations × MusicGen-{small,large}.

## Layout

```
per_benchmark/{small,large}/{musiceval,songeval,aime,musicpref,music_arena}/
    f01_loss_only_cnn_clean_best.pth
    f02_entropy_only_cnn_clean_best.pth
    f03_sae_only_cnn_clean_best.pth
    f04_loss_entropy_cnn_clean_best.pth
    f05_entropy_sae_cnn_clean_best.pth
    f06_loss_sae_cnn_clean_best.pth
    f07_loss_entropy_sae_cnn_clean_best.pth
pooled/{small,large}/          # trained on the union of the five benchmarks
    (same seven files)
```

`f01`–`f07` follow the paper's experiment registry (single-signal, pairwise,
and full hybrids). Per-benchmark models produce Table 1's per-benchmark
columns; `pooled/` models produce the *All benchmarks* column.

## Usage

See the [training/eval scripts](https://github.com/YoEv/MEva) —
`training/hybrid/train_cnn.py` (architecture) and
`scripts/eval/eval_14_experiments.py` (evaluation protocol).

## Citation

```bibtex
@inproceedings{li2026meva,
  title     = {Do Music Generative Models Understand Musical Qualities? Automatic Music Evaluation with Model-Intrinsic Signals},
  author    = {Li, Xiaosha and Liu, Chun and Wang, Ziyu},
  booktitle = {Proc. of the 27th Int. Society for Music Information Retrieval Conf. (ISMIR)},
  year      = {2026}
}
```