Jarbas's picture
Card title: chatterbox-multi-unicode
5d86f18 verified
|
Raw History Blame Contribute Delete
3.68 kB
metadata
license: mit
library_name: phoonnx
pipeline_tag: text-to-speech
language:
  - ar
  - bg
  - cs
  - da
  - de
  - el
  - en
  - es
  - fi
  - fr
  - he
  - hi
  - hu
  - it
  - ja
  - ko
  - ms
  - nl
  - 'no'
  - pl
  - pt
  - ro
  - ru
  - sk
  - sv
  - sw
  - ta
  - tr
  - vi
  - zh
tags:
  - phoonnx
  - chatterbox-multilingual

chatterbox-multi-unicode

A Chatterbox model by Resemble AI, from ResembleAI/chatterbox, packaged for phoonnx, our ONNX text-to-speech library.

Model details

Architecture Chatterbox
Upstream ResembleAI/chatterbox
ONNX conversion onnx-community/chatterbox-multilingual-ONNX
Author Resemble AI
Languages ar, bg-BG, cs-CZ, da-DK, de-DE, el-GR, en-US, es-ES, fi-FI, fr-FR, he-IL, hi-IN, hu-HU, it-IT, ja-JP, ko-KR, ms-MY, nl-NL, no-NO, pl-PL, pt-PT, ro-RO, ru-RU, sk-SK, sv-SE, sw, ta-IN, tr-TR, vi-VN, zh-CN
Text input Unicode codepoints of the text

How to use

pip install --pre phoonnx
import wave
from phoonnx.model_manager import TTSModelManager

manager = TTSModelManager()
manager.load()
manager.merge_default_voices()

voice = manager.voices["chatterbox/multilingual/ar"].load()

with wave.open("output.wav", "wb") as wav_file:
    text = "..."  # a sentence in ar
    voice.synthesize_wav(text, wav_file)

The voice clones a speaker from reference audio; the phoonnx README shows how to pass it.

In OVOS, phoonnx installs the ovos-tts-plugin-phoonnx plugin. Select the voice in mycroft.conf:

{"tts": {"module": "ovos-tts-plugin-phoonnx", "ovos-tts-plugin-phoonnx": {"voice": "chatterbox/multilingual/ar"}}}

Text front end

phoonnx hands the text to scriptconv, our text conversion library, which normalises it and turns it into the model's input:

  • unicode: UnicodeCodepointPhonemizer from scriptconv.phonemizers.base, in scriptconv itself.

phoonnx voices

  • chatterbox/multilingual/ar
  • chatterbox/multilingual/bg
  • chatterbox/multilingual/cs
  • chatterbox/multilingual/da
  • chatterbox/multilingual/de
  • chatterbox/multilingual/el
  • chatterbox/multilingual/en
  • chatterbox/multilingual/es
  • chatterbox/multilingual/fi
  • chatterbox/multilingual/fr
  • chatterbox/multilingual/he
  • chatterbox/multilingual/hi
  • chatterbox/multilingual/hu
  • chatterbox/multilingual/it
  • chatterbox/multilingual/ja
  • chatterbox/multilingual/ko
  • chatterbox/multilingual/ms
  • chatterbox/multilingual/nl
  • chatterbox/multilingual/no
  • chatterbox/multilingual/pl
  • chatterbox/multilingual/pt
  • chatterbox/multilingual/ro
  • chatterbox/multilingual/ru
  • chatterbox/multilingual/sk
  • chatterbox/multilingual/sv
  • chatterbox/multilingual/sw
  • chatterbox/multilingual/ta
  • chatterbox/multilingual/tr
  • chatterbox/multilingual/vi
  • chatterbox/multilingual/zh

Licence

The upstream declares the licence mit in the model card of ResembleAI/chatterbox.

Related projects

  • phoonnx: runs this model, in Python and in OVOS.
  • scriptconv: text normalisation, script conversion and phonemizers for phoonnx.
  • The Chatterbox collection: every Chatterbox model packaged for phoonnx.