Text-to-Speech
ONNX
F5-TTS
phoonnx
Jarbas's picture
Add the voice and its card
29b93a4 verified
|
Raw History Blame Contribute Delete
4.68 kB
metadata
license: apache-2.0
language:
  - as
  - bn
  - bo
  - gu
  - hi
  - kn
  - ks
  - kok
  - mai
  - ml
  - mni
  - mr
  - ne
  - or
  - pa
  - sa
  - sat
  - sd
  - ta
  - te
  - ur
  - doi
  - raj
  - en
base_model:
  - SPRINGLab/SPRING_F5
base_model_relation: quantized
library_name: phoonnx
pipeline_tag: text-to-speech
tags:
  - phoonnx
  - onnx
  - f5-tts
  - text-to-speech
datasets:
  - ai4bharat/IndicVoices
  - ai4bharat/Rasa

f5tts-multi-spring-unicode

F5-TTS voice for phoonnx, converted to ONNX from SPRINGLab/SPRING_F5.

Model details

Field Value
Repository Phoonnx/f5tts-multi-spring-unicode
Voice id f5tts/spring
Languages mul
Architecture F5-TTS
Text input Unicode codepoints of the text
Punctuation Trained with the clause punctuation , . ! ? : ;. phoonnx 1.96.1a1 and later keep it at synthesis.
Sample rate 24000 Hz
Speakers 1
Training code Upstream training code; see SPRINGLab/SPRING_F5
Phonemizer None: the model reads the Unicode codepoints of the text, NFD-decomposed by phoonnx
Training data ai4bharat/IndicVoices, ai4bharat/Rasa
Base model SWivid/F5-TTS
Training checkpoint None in this repository
Upstream SPRINGLab/SPRING_F5
Trained by SPRINGLab (upstream)
Licence Apache 2.0

QA method: ASR nemo-parakeet-tdt-0.6b-v3 int8. Sentences: FLORES-200 devtest, 3 sentences. The reference clip is LJ Speech LJ001-0009 (public domain). QA covers English only. The voice config maps single characters only, so phoonnx gives the model one id for each character, as F5-TTS does for text that is not Chinese. phoonnx drops the danda (।) at the end of a Hindi sentence.

Check fp32 int8
Result pass pass
WER 0.000 0.000
Characters per second 13.8 to 13.8 -
Length, int8 / fp32 - 1.00 to 1.00
Graph size 1455.0 MB 463.1 MB
Peak RSS 3201.0 MB 1193.8 MB
RTF (median of 3, 4 threads) 31.127 19.886

Measured on AMD Ryzen 5 7600 6-Core Processor. RTF is synthesis time divided by audio length.

Each <graph>_int8.onnx file is the int8 copy of <graph>.onnx, made with onnxruntime.quantization.quantize_dynamic (onnxruntime 1.32.0) and QInt8 weights. The copy passes the int8 gate: its audio is finite and not silent, its length is 0.8 to 1.25 of the fp32 length, and its WER is at most the fp32 WER + 0.05.

How to use

Install phoonnx:

pip install --pre phoonnx

Then run:

import wave
from phoonnx.config import SynthesisConfig
from phoonnx.model_manager import TTSModelManager

manager = TTSModelManager()
manager.load()
manager.merge_default_voices()

voice = manager.voices["f5tts/spring"].load()

with wave.open("output.wav", "wb") as wav_file:
    # The voice clones the speaker of a short clip. Give the clip and its transcription.
    syn_config = SynthesisConfig(speaker_reference="reference.wav",
                                 speaker_reference_text="What the speaker says in the clip.")
    voice.synthesize_wav("All citizens of Vatican City are Roman Catholic.", wav_file, syn_config=syn_config)

In OVOS, phoonnx installs the ovos-tts-plugin-phoonnx plugin. Select this voice in mycroft.conf:

{
  "tts": {
    "module": "ovos-tts-plugin-phoonnx",
    "ovos-tts-plugin-phoonnx": {
      "voice": "f5tts/spring"
    }
  }
}

This voice is not for commercial use. See "Ownership and licensing".

Training data

Trained on ai4bharat/IndicVoices, ai4bharat/Rasa.

Related links

Ownership and licensing

SPRINGLab trained this model and publishes it at SPRINGLab/SPRING_F5 under Apache 2.0. TigreGotico Lda converted it to ONNX for phoonnx and publishes the conversion under the same licence. The conversion does not change the weights; the int8 copies quantize them. Credit SPRINGLab when you use this voice. SPRINGLab fine-tuned this model from SWivid/F5-TTS, which is CC BY-NC 4.0. Commercial use of this voice is therefore not allowed.