license: mit
tags:
- music
- music-evaluation
- musicgen
- audio
library_name: pytorch
MEva evaluator checkpoints
Trained CNN evaluators for the ISMIR 2026 paper "Do Music Generative Models Understand Musical Qualities? Automatic Music Evaluation with Model-Intrinsic Signals" (Xiaosha Li, Chun Liu, Ziyu Wang).
Each checkpoint maps intrinsic signals of a frozen MusicGen (per-token loss, predictive entropy, and SAE latents under teacher forcing, delay pattern) to a human quality rating. 14 variants: 7 signal combinations × MusicGen-{small,large}.
Layout
per_benchmark/{small,large}/{musiceval,songeval,aime,musicpref,music_arena}/
f01_loss_only_cnn_clean_best.pth
f02_entropy_only_cnn_clean_best.pth
f03_sae_only_cnn_clean_best.pth
f04_loss_entropy_cnn_clean_best.pth
f05_entropy_sae_cnn_clean_best.pth
f06_loss_sae_cnn_clean_best.pth
f07_loss_entropy_sae_cnn_clean_best.pth
pooled/{small,large}/ # trained on the union of the five benchmarks
(same seven files)
f01–f07 follow the paper's experiment registry (single-signal, pairwise,
and full hybrids). Per-benchmark models produce Table 1's per-benchmark
columns; pooled/ models produce the All benchmarks column.
Usage
See the training/eval scripts —
training/hybrid/train_cnn.py (architecture) and
scripts/eval/eval_14_experiments.py (evaluation protocol).
Citation
@inproceedings{li2026meva,
title = {Do Music Generative Models Understand Musical Qualities? Automatic Music Evaluation with Model-Intrinsic Signals},
author = {Li, Xiaosha and Liu, Chun and Wang, Ziyu},
booktitle = {Proc. of the 27th Int. Society for Music Information Retrieval Conf. (ISMIR)},
year = {2026}
}