---
license: mit
pipeline_tag: audio-to-audio
tags:
  - audio
  - speech
  - vocal-intensity
  - speech-synthesis
---

# Vocal intensity conversion with continuous SPL control via adversarial training

Quentin Le Tellier, Albert Rilliard, Olivier Perrotin, Marc Evrard
Submitted to ICASSP 2027.

**[Code](https://github.com/vers-project/vocal-intensity-conversion)** ·
**[Audio samples](https://vers-project.github.io/vocal-intensity-conversion/)**

These are the trained converters of the paper. A converter renders a recorded
utterance at a target vocal intensity, given in dB SPL at 1 m, while preserving
its content and speaker identity. It transforms the features of a frozen speech
encoder, and a pretrained vocoder resynthesizes the speech.

## Models

| folder | encoder/decoder pair | parameters |
|---|---|---|
| `converter-wavlm` | WavLM-Large layer 6 + kNN-VC HiFi-GAN | 2.66 M |

Each folder holds the converter weights (`model.safetensors`) and the
configuration needed to rebuild it (`config.yaml`). The encoder and vocoder are
not included: the code downloads them from their original sources.

## Usage

```bash
git clone https://github.com/vers-project/vocal-intensity-conversion.git
cd vocal-intensity-conversion
uv run --extra cpu --extra hub scripts/convert.py \
    --model     vers-project/vocal-intensity-conversion \
    --subfolder converter-wavlm \
    --input     speech.wav \
    --target-db 70.0 \
    --output    speech_70dB.wav
```

The first run downloads WavLM-Large and the kNN-VC HiFi-GAN (about 1.3 GB)
into `~/.cache/vic`.

## Training

The converter is a Transformer encoder (six layers, width 128, four heads),
trained as a conditional cycle-consistent GAN without parallel data, on the
training split of [AVID](https://doi.org/10.5281/zenodo.8331897), the Aalto Vocal
Intensity Database (CC BY 4.0). It was trained for 11 400 iterations, as reported
in the paper.

## Limitations

The converter was trained on AVID only. Target intensities outside the range
seen in training, 42.8 to 76.1 dB SPL, were not evaluated.

## Third-party components

- WavLM-Large (Chen et al., 2022) and the HiFi-GAN vocoder of kNN-VC
  (Baas et al., 2023), both from https://github.com/bshall/knn-vc (MIT).

## Citation

```bibtex
@misc{letellier2027vocal,
  title  = {Vocal Intensity Conversion with Continuous {SPL} Control via Adversarial Training},
  author = {Le Tellier, Quentin and Rilliard, Albert and Perrotin, Olivier and Evrard, Marc},
  year   = {2026},
  note   = {Submitted to ICASSP 2027}
}
```

## Licence

MIT.