File size: 3,679 Bytes
7523ad7
 
d380f10
 
7523ad7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
d380f10
7523ad7
 
 
 
 
 
 
 
 
 
 
 
 
d380f10
7523ad7
 
5d86f18
7523ad7
d380f10
7523ad7
d380f10
7523ad7
d380f10
 
 
 
 
 
 
 
7523ad7
d380f10
7523ad7
d380f10
 
 
7523ad7
 
 
 
 
 
 
 
 
d380f10
 
 
 
 
7523ad7
 
d380f10
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7523ad7
d380f10
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
---
license: mit
library_name: phoonnx
pipeline_tag: text-to-speech
language:
- ar
- bg
- cs
- da
- de
- el
- en
- es
- fi
- fr
- he
- hi
- hu
- it
- ja
- ko
- ms
- nl
- no
- pl
- pt
- ro
- ru
- sk
- sv
- sw
- ta
- tr
- vi
- zh
tags:
- phoonnx
- chatterbox-multilingual
---

# chatterbox-multi-unicode

A Chatterbox model by Resemble AI, from [ResembleAI/chatterbox](https://huggingface.co/ResembleAI/chatterbox), packaged for [phoonnx](https://github.com/TigreGotico/phoonnx), our ONNX text-to-speech library.

## Model details

| | |
|---|---|
| Architecture | [Chatterbox](https://huggingface.co/collections/Phoonnx/chatterbox-6af8ea77fad812f585a5c864) |
| Upstream | [ResembleAI/chatterbox](https://huggingface.co/ResembleAI/chatterbox) |
| ONNX conversion | [onnx-community/chatterbox-multilingual-ONNX](https://huggingface.co/onnx-community/chatterbox-multilingual-ONNX) |
| Author | Resemble AI |
| Languages | `ar`, `bg-BG`, `cs-CZ`, `da-DK`, `de-DE`, `el-GR`, `en-US`, `es-ES`, `fi-FI`, `fr-FR`, `he-IL`, `hi-IN`, `hu-HU`, `it-IT`, `ja-JP`, `ko-KR`, `ms-MY`, `nl-NL`, `no-NO`, `pl-PL`, `pt-PT`, `ro-RO`, `ru-RU`, `sk-SK`, `sv-SE`, `sw`, `ta-IN`, `tr-TR`, `vi-VN`, `zh-CN` |
| Text input | Unicode codepoints of the text |

## How to use

```bash
pip install --pre phoonnx
```

```python
import wave
from phoonnx.model_manager import TTSModelManager

manager = TTSModelManager()
manager.load()
manager.merge_default_voices()

voice = manager.voices["chatterbox/multilingual/ar"].load()

with wave.open("output.wav", "wb") as wav_file:
    text = "..."  # a sentence in ar
    voice.synthesize_wav(text, wav_file)
```

The voice clones a speaker from reference audio; the phoonnx README shows how to pass it.

In OVOS, phoonnx installs the `ovos-tts-plugin-phoonnx` plugin. Select the voice in `mycroft.conf`:

```json
{"tts": {"module": "ovos-tts-plugin-phoonnx", "ovos-tts-plugin-phoonnx": {"voice": "chatterbox/multilingual/ar"}}}
```

## Text front end

phoonnx hands the text to [scriptconv](https://github.com/TigreGotico/scriptconv), our text conversion library, which normalises it and turns it into the model's input:

- `unicode`: `UnicodeCodepointPhonemizer` from `scriptconv.phonemizers.base`, in scriptconv itself.

## phoonnx voices

- `chatterbox/multilingual/ar`
- `chatterbox/multilingual/bg`
- `chatterbox/multilingual/cs`
- `chatterbox/multilingual/da`
- `chatterbox/multilingual/de`
- `chatterbox/multilingual/el`
- `chatterbox/multilingual/en`
- `chatterbox/multilingual/es`
- `chatterbox/multilingual/fi`
- `chatterbox/multilingual/fr`
- `chatterbox/multilingual/he`
- `chatterbox/multilingual/hi`
- `chatterbox/multilingual/hu`
- `chatterbox/multilingual/it`
- `chatterbox/multilingual/ja`
- `chatterbox/multilingual/ko`
- `chatterbox/multilingual/ms`
- `chatterbox/multilingual/nl`
- `chatterbox/multilingual/no`
- `chatterbox/multilingual/pl`
- `chatterbox/multilingual/pt`
- `chatterbox/multilingual/ro`
- `chatterbox/multilingual/ru`
- `chatterbox/multilingual/sk`
- `chatterbox/multilingual/sv`
- `chatterbox/multilingual/sw`
- `chatterbox/multilingual/ta`
- `chatterbox/multilingual/tr`
- `chatterbox/multilingual/vi`
- `chatterbox/multilingual/zh`

## Licence

The upstream declares the licence `mit` in the model card of ResembleAI/chatterbox.

## Related projects

- [phoonnx](https://github.com/TigreGotico/phoonnx): runs this model, in Python and in OVOS.
- [scriptconv](https://github.com/TigreGotico/scriptconv): text normalisation, script conversion and phonemizers for phoonnx.
- The [Chatterbox](https://huggingface.co/collections/Phoonnx/chatterbox-6af8ea77fad812f585a5c864) collection: every Chatterbox model packaged for phoonnx.