| # WenetSpeech-Min TTS Model |
|
|
| WenetSpeech-Min TTS is a Minnan (Southern Min) text-to-speech model based on |
| [Fun-CosyVoice3-0.5B-2512](https://huggingface.co/FunAudioLLM/Fun-CosyVoice3-0.5B-2512). |
| The CosyVoice3 LLM is fine-tuned for Minnan speech synthesis, while the Flow, |
| HiFT, speaker encoder, speech tokenizer, and text tokenizer are inherited from |
| the base model. |
|
|
| ## Install |
|
|
| ### Clone the repo |
|
|
| Clone the CosyVoice repository and initialize its submodules: |
|
|
| ```bash |
| git clone https://github.com/ASLP-lab/WenetSpeech-Min-Repo.git |
| |
| cd CosyVoice3-Min |
| ``` |
|
|
| ### Create Conda env |
|
|
| ```bash |
| conda create -n cosyvoice3 -y python=3.10 |
| conda activate cosyvoice3 |
| pip install -r requirements.txt |
| ``` |
|
|
| ### Model download |
|
|
| Download the complete checkpoint from Hugging Face: |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| |
| snapshot_download( |
| repo_id="ASLP-lab/WS-MIN-TTS", |
| local_dir="pretrained_models/WS-MIN-TTS", |
| ) |
| ``` |
|
|
| The CosyVoice3 model files will be downloaded to |
| `pretrained_models/WS-MIN-TTS/CosyVoice3-Min/`. |
|
|
| ## Usage |
|
|
| The following example performs zero-shot Minnan speech synthesis using a |
| reference audio file: |
|
|
| ```python |
| import os |
| import sys |
| |
| import soundfile as sf |
| |
| sys.path.append("third_party/Matcha-TTS") |
| |
| from cosyvoice.cli.cosyvoice import AutoModel |
| |
| |
| model_dir = "pretrained_models/WS-MIN-TTS/CosyVoice3-Min" |
| prompt_wav = "prompt.wav" |
| output_wav = "output.wav" |
| |
| if not os.path.isfile(prompt_wav): |
| raise FileNotFoundError(f"Reference audio not found: {prompt_wav}") |
| |
| cosyvoice = AutoModel(model_dir=model_dir) |
| instruct_text = "用闽南语说这句话<|endofprompt|>" |
| tts_text = "你欲食啥物？" |
| |
| for model_output in cosyvoice.inference_instruct2( |
| tts_text, |
| instruct_text, |
| prompt_wav, |
| stream=False, |
| ): |
| speech = model_output["tts_speech"].squeeze(0).cpu().float().numpy() |
| sf.write(output_wav, speech, cosyvoice.sample_rate) |
| |
| print(f"Generated audio saved to: {output_wav}") |
| ``` |
|
|
| Use a clean reference recording for `prompt.wav`. The generated audio is saved |
| to `output.wav`. |
|
|
