Jarbas's picture
Card title: chatterbox-multi-unicode
5d86f18 verified
|
Raw History Blame Contribute Delete
3.68 kB
---
license: mit
library_name: phoonnx
pipeline_tag: text-to-speech
language:
- ar
- bg
- cs
- da
- de
- el
- en
- es
- fi
- fr
- he
- hi
- hu
- it
- ja
- ko
- ms
- nl
- no
- pl
- pt
- ro
- ru
- sk
- sv
- sw
- ta
- tr
- vi
- zh
tags:
- phoonnx
- chatterbox-multilingual
---
# chatterbox-multi-unicode
A Chatterbox model by Resemble AI, from [ResembleAI/chatterbox](https://huggingface.co/ResembleAI/chatterbox), packaged for [phoonnx](https://github.com/TigreGotico/phoonnx), our ONNX text-to-speech library.
## Model details
| | |
|---|---|
| Architecture | [Chatterbox](https://huggingface.co/collections/Phoonnx/chatterbox-6af8ea77fad812f585a5c864) |
| Upstream | [ResembleAI/chatterbox](https://huggingface.co/ResembleAI/chatterbox) |
| ONNX conversion | [onnx-community/chatterbox-multilingual-ONNX](https://huggingface.co/onnx-community/chatterbox-multilingual-ONNX) |
| Author | Resemble AI |
| Languages | `ar`, `bg-BG`, `cs-CZ`, `da-DK`, `de-DE`, `el-GR`, `en-US`, `es-ES`, `fi-FI`, `fr-FR`, `he-IL`, `hi-IN`, `hu-HU`, `it-IT`, `ja-JP`, `ko-KR`, `ms-MY`, `nl-NL`, `no-NO`, `pl-PL`, `pt-PT`, `ro-RO`, `ru-RU`, `sk-SK`, `sv-SE`, `sw`, `ta-IN`, `tr-TR`, `vi-VN`, `zh-CN` |
| Text input | Unicode codepoints of the text |
## How to use
```bash
pip install --pre phoonnx
```
```python
import wave
from phoonnx.model_manager import TTSModelManager
manager = TTSModelManager()
manager.load()
manager.merge_default_voices()
voice = manager.voices["chatterbox/multilingual/ar"].load()
with wave.open("output.wav", "wb") as wav_file:
text = "..." # a sentence in ar
voice.synthesize_wav(text, wav_file)
```
The voice clones a speaker from reference audio; the phoonnx README shows how to pass it.
In OVOS, phoonnx installs the `ovos-tts-plugin-phoonnx` plugin. Select the voice in `mycroft.conf`:
```json
{"tts": {"module": "ovos-tts-plugin-phoonnx", "ovos-tts-plugin-phoonnx": {"voice": "chatterbox/multilingual/ar"}}}
```
## Text front end
phoonnx hands the text to [scriptconv](https://github.com/TigreGotico/scriptconv), our text conversion library, which normalises it and turns it into the model's input:
- `unicode`: `UnicodeCodepointPhonemizer` from `scriptconv.phonemizers.base`, in scriptconv itself.
## phoonnx voices
- `chatterbox/multilingual/ar`
- `chatterbox/multilingual/bg`
- `chatterbox/multilingual/cs`
- `chatterbox/multilingual/da`
- `chatterbox/multilingual/de`
- `chatterbox/multilingual/el`
- `chatterbox/multilingual/en`
- `chatterbox/multilingual/es`
- `chatterbox/multilingual/fi`
- `chatterbox/multilingual/fr`
- `chatterbox/multilingual/he`
- `chatterbox/multilingual/hi`
- `chatterbox/multilingual/hu`
- `chatterbox/multilingual/it`
- `chatterbox/multilingual/ja`
- `chatterbox/multilingual/ko`
- `chatterbox/multilingual/ms`
- `chatterbox/multilingual/nl`
- `chatterbox/multilingual/no`
- `chatterbox/multilingual/pl`
- `chatterbox/multilingual/pt`
- `chatterbox/multilingual/ro`
- `chatterbox/multilingual/ru`
- `chatterbox/multilingual/sk`
- `chatterbox/multilingual/sv`
- `chatterbox/multilingual/sw`
- `chatterbox/multilingual/ta`
- `chatterbox/multilingual/tr`
- `chatterbox/multilingual/vi`
- `chatterbox/multilingual/zh`
## Licence
The upstream declares the licence `mit` in the model card of ResembleAI/chatterbox.
## Related projects
- [phoonnx](https://github.com/TigreGotico/phoonnx): runs this model, in Python and in OVOS.
- [scriptconv](https://github.com/TigreGotico/scriptconv): text normalisation, script conversion and phonemizers for phoonnx.
- The [Chatterbox](https://huggingface.co/collections/Phoonnx/chatterbox-6af8ea77fad812f585a5c864) collection: every Chatterbox model packaged for phoonnx.