Hugging Face's logo

ASLP-lab
/
WS-MIN-TTS

ONNX
Safetensors
Model card Files
xet
 Community
WS-MIN-TTS
File size: 2,036 Bytes
ec60f80
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
# WenetSpeech-Min TTS Model

WenetSpeech-Min TTS is a Minnan (Southern Min) text-to-speech model based on
[Fun-CosyVoice3-0.5B-2512](https://huggingface.co/FunAudioLLM/Fun-CosyVoice3-0.5B-2512).
The CosyVoice3 LLM is fine-tuned for Minnan speech synthesis, while the Flow,
HiFT, speaker encoder, speech tokenizer, and text tokenizer are inherited from
the base model.

## Install

### Clone the repo

Clone the CosyVoice repository and initialize its submodules:

```bash
git clone https://github.com/ASLP-lab/WenetSpeech-Min-Repo.git

cd CosyVoice3-Min
```

### Create Conda env

```bash
conda create -n cosyvoice3 -y python=3.10
conda activate cosyvoice3
pip install -r requirements.txt
```

### Model download

Download the complete checkpoint from Hugging Face:

```python
from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="ASLP-lab/WS-MIN-TTS",
    local_dir="pretrained_models/WS-MIN-TTS",
)
```

The CosyVoice3 model files will be downloaded to
`pretrained_models/WS-MIN-TTS/CosyVoice3-Min/`.

## Usage

The following example performs zero-shot Minnan speech synthesis using a
reference audio file:

```python
import os
import sys

import soundfile as sf

sys.path.append("third_party/Matcha-TTS")

from cosyvoice.cli.cosyvoice import AutoModel


model_dir = "pretrained_models/WS-MIN-TTS/CosyVoice3-Min"
prompt_wav = "prompt.wav"
output_wav = "output.wav"

if not os.path.isfile(prompt_wav):
    raise FileNotFoundError(f"Reference audio not found: {prompt_wav}")

cosyvoice = AutoModel(model_dir=model_dir)
instruct_text = "用闽南语说这句话<|endofprompt|>"
tts_text = "你欲食啥物？"

for model_output in cosyvoice.inference_instruct2(
    tts_text,
    instruct_text,
    prompt_wav,
    stream=False,
):
    speech = model_output["tts_speech"].squeeze(0).cpu().float().numpy()
    sf.write(output_wav, speech, cosyvoice.sample_rate)

print(f"Generated audio saved to: {output_wav}")
```

Use a clean reference recording for `prompt.wav`. The generated audio is saved
to `output.wav`.