| --- |
| license: cc-by-nc-4.0 |
| base_model: |
| - SWivid/F5-TTS |
| tags: |
| - speech-enhancement |
| - f5-tts |
| - controlnet |
| - qwen3-asr |
| --- |
| |
| # ReConSE |
|
|
| ## Model Summary |
|
|
| ReConSE: Universal speech enhancement via recurrent control over a pretrained F5-TTS model. |
|
|
| ## Model Weights |
|
|
| Two checkpoint files are provided: |
|
|
| - **`model_70000_inference.pt`** — Full inference checkpoint. It contains the EMA weights for the complete ReConSE model: the F5-TTS v1 Base backbone, Recurrent ControlNet, and Qwen audio adapters. It excludes optimizer and scheduler states and the non-EMA training weights. The Qwen audio encoder and Vocos vocoder are not included. |
|
|
| - **`model_70000_controlnet_adapter.pt`** — Compact EMA weights for the Recurrent ControlNet and Qwen audio adapters only. It does not contain the F5-TTS backbone, so it is **not a standalone checkpoint**. |
| |
| ## Required External Assets |
| |
| Both checkpoint variants require the ReConSE code and configuration from the [GitHub repository](https://github.com/Lin-Johnson/ReConSE), including the model configuration and vocabulary file at `src/f5_tts/infer/examples/vocab.txt`. |
| |
| Because both checkpoints use Qwen audio adapters, download the [Qwen3-ASR-1.7B model](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) for its audio encoder. The Qwen encoder weights are not bundled in either ReConSE checkpoint. |
| |
| The Vocos vocoder is also required to convert the generated mel spectrograms into audio. Download [Vocos mel-24kHz](https://huggingface.co/charactr/vocos-mel-24khz). |
| |
| ### Using `model_70000_inference.pt` |
| |
| This checkpoint already contains the F5-TTS backbone weights. Load it with the ReConSE inference code and provide the external Qwen encoder and Vocos assets through the configuration. |
| |
| ### Using `model_70000_controlnet_adapter.pt` |
| |
| This checkpoint requires the [F5-TTS v1 Base model weights](https://huggingface.co/SWivid/F5-TTS/tree/main/F5TTS_v1_Base), including `model_1250000.safetensors`. |
| |
| The current inference loader constructs the model architecture and then loads the supplied checkpoint. Since this file contains only ControlNet and adapter weights, using it directly would leave the F5-TTS backbone uninitialized. Modify the loading code to: |
| |
| 1. Initialize the F5-TTS v1 Base backbone from its pretrained checkpoint. |
| 2. Load the ReConSE ControlNet and adapter weights on top of that backbone. |
| |
| The vocabulary used to construct the model must match the vocabulary expected by the checkpoint. |
| |
| ## Code |
| |
| [GitHub repository](https://github.com/Lin-Johnson/ReConSE) |
