| --- |
| license: mit |
| library_name: pytorch |
| tags: |
| - speech-editing |
| - speech-synthesis |
| - text-to-speech |
| --- |
| |
| # wavepainter |
|
|
| Released checkpoints for [wavepainter](https://github.com/pujariaditya/wavepainter), |
| a masked-span speech editor. |
|
|
| On the English `full` split of Ming-Freeform-Audio-Edit, against Ren et al. |
| ([arXiv:2602.00560](https://arxiv.org/abs/2602.00560), the row with GRPO): |
|
|
| | edit type | metric | Ren et al. | wavepainter | |
| |---|---|---|---| |
| | substitution | WER | 4.41 | **3.063** | |
| | insertion | WER | 4.97 | **4.128** | |
| | deletion | WER | **6.88** | 9.136 | |
| | substitution | SIM | 0.78 | **0.943** | |
| | insertion | SIM | 0.82 | **0.961** | |
| | deletion | SIM | 0.78 | **0.918** | |
| | substitution | DNSMOS | **3.11** | 2.931 | |
| | insertion | DNSMOS | **3.18** | 2.905 | |
| | deletion | DNSMOS | **3.09** | 2.938 | |
|
|
| Five of the nine numbers they report improve on theirs: substitution and |
| insertion WER, and speaker similarity on all three edit types. We do not beat |
| them on deletion WER or on DNSMOS anywhere -- see the repository's Limitations. |
|
|
| ## Files |
|
|
| - `phase1-base/` — the phase-1 editor. Start here to re-run phase 2 (~1 GPU-hour). |
| - `phase2-hubert-a085/` — the released model. This produces the numbers above. |
|
|
| ## Use |
|
|
| ```bash |
| git clone https://github.com/pujariaditya/wavepainter && cd wavepainter |
| pip install -e . && ./setup.sh |
| python scripts/download_weights.py |
| ./scripts/verify_benchmark.sh |
| ``` |
|
|
| Checkpoints are sha256-pinned in `scripts/download_weights.py`. Licence and |
| attribution, including for the components these weights derive from, are in the |
| repository's `ATTRIBUTION.md`. |
|
|
