|
Download README.md from vers-project/vocal-intensity-conversion: direct link, hf CLI and curl.
- Browser
- Download file 2.55 kB
-
https://huggingface.co/vers-project/vocal-intensity-conversion/resolve/main/README.md
- Command line
-
hf download hf://vers-project/vocal-intensity-conversion/README.md
-
curl -L -o README.md https://huggingface.co/vers-project/vocal-intensity-conversion/resolve/main/README.md
2.55 kB
| license: mit | |
| pipeline_tag: audio-to-audio | |
| tags: | |
| - audio | |
| - speech | |
| - vocal-intensity | |
| - speech-synthesis | |
| # Vocal intensity conversion with continuous SPL control via adversarial training | |
| Quentin Le Tellier, Albert Rilliard, Olivier Perrotin, Marc Evrard | |
| Submitted to ICASSP 2027. | |
| **[Code](https://github.com/vers-project/vocal-intensity-conversion)** · | |
| **[Audio samples](https://vers-project.github.io/vocal-intensity-conversion/)** | |
| These are the trained converters of the paper. A converter renders a recorded | |
| utterance at a target vocal intensity, given in dB SPL at 1 m, while preserving | |
| its content and speaker identity. It transforms the features of a frozen speech | |
| encoder, and a pretrained vocoder resynthesizes the speech. | |
| ## Models | |
| | folder | encoder/decoder pair | parameters | | |
| |---|---|---| | |
| | `converter-wavlm` | WavLM-Large layer 6 + kNN-VC HiFi-GAN | 2.66 M | | |
| Each folder holds the converter weights (`model.safetensors`) and the | |
| configuration needed to rebuild it (`config.yaml`). The encoder and vocoder are | |
| not included: the code downloads them from their original sources. | |
| ## Usage | |
| ```bash | |
| git clone https://github.com/vers-project/vocal-intensity-conversion.git | |
| cd vocal-intensity-conversion | |
| uv run --extra cpu --extra hub scripts/convert.py \ | |
| --model vers-project/vocal-intensity-conversion \ | |
| --subfolder converter-wavlm \ | |
| --input speech.wav \ | |
| --target-db 70.0 \ | |
| --output speech_70dB.wav | |
| ``` | |
| The first run downloads WavLM-Large and the kNN-VC HiFi-GAN (about 1.3 GB) | |
| into `~/.cache/vic`. | |
| ## Training | |
| The converter is a Transformer encoder (six layers, width 128, four heads), | |
| trained as a conditional cycle-consistent GAN without parallel data, on the | |
| training split of [AVID](https://doi.org/10.5281/zenodo.8331897), the Aalto Vocal | |
| Intensity Database (CC BY 4.0). It was trained for 11 400 iterations, as reported | |
| in the paper. | |
| ## Limitations | |
| The converter was trained on AVID only. Target intensities outside the range | |
| seen in training, 42.8 to 76.1 dB SPL, were not evaluated. | |
| ## Third-party components | |
| - WavLM-Large (Chen et al., 2022) and the HiFi-GAN vocoder of kNN-VC | |
| (Baas et al., 2023), both from https://github.com/bshall/knn-vc (MIT). | |
| ## Citation | |
| ```bibtex | |
| @misc{letellier2027vocal, | |
| title = {Vocal Intensity Conversion with Continuous {SPL} Control via Adversarial Training}, | |
| author = {Le Tellier, Quentin and Rilliard, Albert and Perrotin, Olivier and Evrard, Marc}, | |
| year = {2026}, | |
| note = {Submitted to ICASSP 2027} | |
| } | |
| ``` | |
| ## Licence | |
| MIT. | |