Instructions to use JohnsonLin0216/ReConSE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- F5-TTS
How to use JohnsonLin0216/ReConSE with F5-TTS:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -15,6 +15,37 @@ tags:
|
|
| 15 |
|
| 16 |
ReConSE: Universal speech enhancement via recurrent control over a pretrained F5-TTS model.
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
## Code
|
| 19 |
|
| 20 |
[GitHub repository](https://github.com/Lin-Johnson/ReConSE)
|
|
|
|
| 15 |
|
| 16 |
ReConSE: Universal speech enhancement via recurrent control over a pretrained F5-TTS model.
|
| 17 |
|
| 18 |
+
## Model Weights
|
| 19 |
+
|
| 20 |
+
Two checkpoint files are provided:
|
| 21 |
+
|
| 22 |
+
- **`model_70000_inference.pt`** — Full inference checkpoint. It contains the EMA weights for the complete ReConSE model: the F5-TTS v1 Base backbone, Recurrent ControlNet, and Qwen audio adapters. It excludes optimizer and scheduler states and the non-EMA training weights. The Qwen audio encoder and Vocos vocoder are not included.
|
| 23 |
+
|
| 24 |
+
- **`model_70000_controlnet_adapter.pt`** — Compact EMA weights for the Recurrent ControlNet and Qwen audio adapters only. It does not contain the F5-TTS backbone, so it is **not a standalone checkpoint**.
|
| 25 |
+
|
| 26 |
+
## Required External Assets
|
| 27 |
+
|
| 28 |
+
Both checkpoint variants require the ReConSE code and configuration from the [GitHub repository](https://github.com/Lin-Johnson/ReConSE), including the model configuration and vocabulary file at `src/f5_tts/infer/examples/vocab.txt`.
|
| 29 |
+
|
| 30 |
+
Because both checkpoints use Qwen audio adapters, download the [Qwen3-ASR-1.7B model](https://huggingface.co/Qwen/Qwen3-ASR-1.7B) for its audio encoder. The Qwen encoder weights are not bundled in either ReConSE checkpoint.
|
| 31 |
+
|
| 32 |
+
The Vocos vocoder is also required to convert the generated mel spectrograms into audio. Download [Vocos mel-24kHz](https://huggingface.co/charactr/vocos-mel-24khz).
|
| 33 |
+
|
| 34 |
+
### Using `model_70000_inference.pt`
|
| 35 |
+
|
| 36 |
+
This checkpoint already contains the F5-TTS backbone weights. Load it with the ReConSE inference code and provide the external Qwen encoder and Vocos assets through the configuration.
|
| 37 |
+
|
| 38 |
+
### Using `model_70000_controlnet_adapter.pt`
|
| 39 |
+
|
| 40 |
+
This checkpoint requires the [F5-TTS v1 Base model weights](https://huggingface.co/SWivid/F5-TTS/tree/main/F5TTS_v1_Base), including `model_1250000.safetensors`.
|
| 41 |
+
|
| 42 |
+
The current inference loader constructs the model architecture and then loads the supplied checkpoint. Since this file contains only ControlNet and adapter weights, using it directly would leave the F5-TTS backbone uninitialized. Modify the loading code to:
|
| 43 |
+
|
| 44 |
+
1. Initialize the F5-TTS v1 Base backbone from its pretrained checkpoint.
|
| 45 |
+
2. Load the ReConSE ControlNet and adapter weights on top of that backbone.
|
| 46 |
+
|
| 47 |
+
The vocabulary used to construct the model must match the vocabulary expected by the checkpoint.
|
| 48 |
+
|
| 49 |
## Code
|
| 50 |
|
| 51 |
[GitHub repository](https://github.com/Lin-Johnson/ReConSE)
|