| --- |
| tags: |
| - audio |
| - music-generation |
| - midi-to-audio |
| - audio-super-resolution |
| --- |
| |
| # CPR: COMBINING GLOBAL COMPOSING, LOCAL PERFORMING AND FULL-SEQUENCE REFINING IN PIANO RENDERING WITH CONTINUOUS AUTOREGRESSIVE MODELLING |
|
|
| 1. **Composer–Performer (CP)** takes prompt audio, prompt MIDI, and target MIDI, |
| and renders **24 kHz mono audio**. The Composer is an autoregressive Qwen3 |
| Transformer; the Performer renders local Mel spectrograms with flow matching. |
| A Vocos vocoder converts the generated Mel spectrograms to audio. |
| 2. **Refiner (R)** takes a **24 kHz audio file** and writes **48 kHz mono audio** |
| using the LavaSR-based bandwidth-extension model and low-frequency fusion. |
|
|
| ## Installation |
|
|
| Create a Conda environment with Python 3.10: |
|
|
| ```bash |
| conda create -n cpr python=3.10 |
| conda activate cpr |
| pip install -r requirements.txt |
| ``` |
|
|
| On the first CP run, CLAP automatically downloads and caches its RoBERTa initialization resources. |
|
|
| ## Checkpoints |
|
|
| Download the inference assets from [Huggingface](https://huggingface.co/FEAfeatherTHER/CPR) into `checkpoints/`. |
|
|
| ```bash |
| hf download FEAfeatherTHER/CPR \ |
| composer_performer.safetensors \ |
| composer_performer_config.json \ |
| vocos.safetensors \ |
| refiner.bin \ |
| --local-dir checkpoints |
| ``` |
|
|
| | Component | File under `checkpoints/` | |
| |---|---| |
| | Composer–Performer | `composer_performer.safetensors` | |
| | Model architecture and Qwen configuration | `composer_performer_config.json` | |
| | 24 kHz Vocos vocoder | `vocos.safetensors` | |
| | Refiner | `refiner.bin` | |
|
|
|
|
| ### Download CLAP |
|
|
| CLAP is required for Composer–Performer inference. Download the official music checkpoint |
| [music_audioset_epoch_15_esc_90.14.pt](https://huggingface.co/lukewys/laion_clap/blob/main/music_audioset_epoch_15_esc_90.14.pt) and save it as `checkpoints/clap.pt`: |
|
|
| ```bash |
| curl --fail --location \ |
| --output checkpoints/clap.pt \ |
| https://huggingface.co/lukewys/laion_clap/resolve/main/music_audioset_epoch_15_esc_90.14.pt |
| ``` |
|
|
| ## Inference |
|
|
| ### 1. Composer–Performer → 24 kHz |
|
|
| ```bash |
| bash scripts/infer_cp.sh |
| ``` |
|
|
| ### 2. Refiner → 48 kHz |
|
|
| use any existing 24 kHz WAV or the output of Composer-Performer: |
|
|
| ```bash |
| python infer_refiner.py --input /path/to/audio_24k.wav --output outputs/refined.wav |
| ``` |
|
|
