Piper & Sherpa-ONNX Speech Models Hub

A production-ready collection of optimized ONNX Text-to-Speech (TTS) models based on the Piper (VITS) architecture. This repository provides pre-trained voices across Vietnamese (vi), English (en), and Indonesian (id), structured with full compatibility for both Piper TTS and Sherpa-ONNX runtimes.

---

Key Highlights

• Neural Architecture: VITS (Variational Inference with Adversarial Learning for End-to-End Text-to-Speech).
• Audio Quality: Native 22,050 Hz sampling rate, 16-bit mono PCM.
• Strict Configuration Integrity: Every single .onnx voice is paired with an exact .onnx.json phoneme configuration file to prevent runtime symbol mismatches.
• Multi-Runtime Ready: Native support for CPU execution, ONNX Runtime GPU (CUDA/DirectML), OpenVINO acceleration, and Sherpa-ONNX embedded deployments.
• Deduplicated & Versioned: Models with identical naming but distinct fine-tuned weights are cleanly separated and indexed (_1, _2).

---

Directory Structure

models/model/
├── piper/
│   ├── voice.json                      # Voice catalog & metadata (44 models)
│   ├── vi/                             # Vietnamese voices (40 models)
│   │   ├── <model_name>.onnx
│   │   └── <model_name>.onnx.json
│   ├── en/                             # English voices (3 models)
│   │   ├── john.onnx (+ .json)
│   │   ├── mattheo.onnx (+ .json)
│   │   └── mattheo1.onnx (+ .json)
│   └── id/                             # Indonesian voices (1 model)
│       ├── goreng.onnx
│       └── goreng.onnx.json
├── sherpa/                             # Sherpa-ONNX converted models (25 voices)
│   ├── tokens.txt                      # Global phoneme tokens symbol table
│   └── <model_name>.onnx
└── README.md

---

Available Voice Models

1. Vietnamese Voices (piper/vi/)

Tổng cộng: 40 giọng tiếng Việt tối ưu.

2. English Voices (piper/en/)

Tổng cộng: 3 giọng tiếng Anh.

3. Indonesian Voices (piper/id/)

Tổng cộng: 1 giọng tiếng Indonesia.

---

Quickstart & Usage

1. Using Piper TTS (Python API)

Install dependencies:

pip install piper-tts

Basic Synthesis to WAV

import wave
from piper import PiperVoice, SynthesisConfig

# Path to model and its paired config
model_path = "piper/vi/banmai.onnx"
config_path = "piper/vi/banmai.onnx.json"

voice = PiperVoice.load(model_path, config_path=config_path)
config = SynthesisConfig(noise_scale=0.667, length_scale=1.0, noise_w=0.8)

text = "Xin chào, đây là giọng đọc thử nghiệm trên hệ thống Piper Text to Speech."

with wave.open("output.wav", "wb") as wav_file:
    voice.synthesize_wav(text, wav_file, syn_config=config)

print("Synthesized successfully to output.wav")

Real-time Audio Streaming

import sounddevice as sd
from piper.voice import PiperVoice, SynthesisConfig

voice = PiperVoice.load("piper/vi/banmai.onnx", config_path="piper/vi/banmai.onnx.json")

text = "Hệ thống đang phát trực tiếp âm thanh từ văn bản."
chunks = voice.synthesize(text)
first_chunk = next(chunks)

stream = sd.OutputStream(
    samplerate=first_chunk.sample_rate,
    channels=first_chunk.sample_channels,
    dtype=first_chunk.audio_float_array.dtype
)
stream.start()

stream.write(first_chunk.audio_float_array)
for chunk in chunks:
    stream.write(chunk.audio_float_array)

stream.stop()
stream.close()

---

2. Using Piper via Command Line (CLI)

# Pipe text directly into piper CLI
echo "Xin chào thế giới" | piper \
  --model piper/vi/banmai.onnx \
  --config piper/vi/banmai.onnx.json \
  --output_file output.wav

---

3. Using Sherpa-ONNX

For embedded systems, mobile devices, or environments where Python bindings for Sherpa-ONNX are preferred:

pip install sherpa-onnx soundfile

import sherpa_onnx as so
import soundfile as sf

tts_config = so.OfflineTtsConfig(
    model=so.OfflineTtsModelConfig(
        vits=so.OfflineTtsVitsModelConfig(
            model="sherpa/banmai.onnx",
            tokens="sherpa/tokens.txt",
            data_dir=""
        )
    )
)

tts = so.OfflineTts(config=tts_config)
audio = tts.generate("Xin chào, đây là giọng đọc từ Sherpa ONNX.")

sf.write("sherpa_output.wav", audio.samples, samplerate=audio.sample_rate, subtype="PCM_16")
print("Saved to sherpa_output.wav")

---

4. Hardware Acceleration (NVIDIA GPU / OpenVINO)

NVIDIA CUDA

To run Piper models on CUDA with ONNX Runtime:

pip install onnxruntime-gpu

Pass use_cuda=True when loading:

voice = PiperVoice.load("piper/vi/banmai.onnx", config_path="piper/vi/banmai.onnx.json", use_cuda=True)

Intel OpenVINO

For Intel CPU/iGPU acceleration:

pip install openvino onnxruntime-openvino

import json
import onnxruntime as ort
from piper import PiperVoice, PiperConfig

with open("piper/vi/banmai.onnx.json", "r", encoding="utf-8") as f:
    piper_config = PiperConfig.from_dict(json.load(f))

session = ort.InferenceSession(
    "piper/vi/banmai.onnx",
    providers=["OpenVINOExecutionProvider"],
    provider_options=[{"device_type": "CPU"}]
)

voice = PiperVoice(config=piper_config, session=session)

---

Technical Specifications

• Input Representation: International Phonetic Alphabet (IPA) tokens mapped via phoneme_id_map (0–255 indices).
• Phonemizer: espeak-ng with voice tags vi (North), vi-vn-x-south (South), en (English), id (Indonesian).
• Inference Latency: Real-Time Factor (RTF) < 0.05 on modern 8-core CPUs, < 0.01 on discrete GPUs.
• Model Size: ~63.5 MB per model (FP32 ONNX graph).

---

Acknowledgments & Credits

• Base neural TTS engine: Rhasspy Piper.
• Vietnamese fine-tuned models: Nghi Studio (NGHI-TTS).
• Sherpa ONNX conversion & engine: Next-gen Kaldi Sherpa-ONNX.

---

License

This repository and model collection are licensed under the Apache-2.0 License.