docker/README.md

BlueTTS 2.5 ONNX Docker (NVIDIA GPU)

GPU image for ONNX Runtime CUDA inference, intended for RunPod GPU pods.

Image: maxme123/blue-v2

What’s inside

• BlueTTS 2.5 ONNX bundle (onnx_slim/, from notmax123/BlueTTS2.5-onnx)
• renikud.onnx — Hebrew G2P from notmax123/RenikudPlus, gender-conditioned
• Five named voices in voices/ — see below
• onnxruntime-gpu with CUDAExecutionProvider
• RunPod serverless handler (docker/rp_handler.py) — default
• Gradio UI on port 7860 when BLUE_SERVE_MODE=gradio

Serverless API

curl -X POST "https://api.runpod.ai/v2/<ENDPOINT_ID>/runsync" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <API_KEY>" \
  -d '{
    "input": {
      "prompt": "Hello from BlueTTS.",
      "voice": "Daniel",
      "lang": "en",
      "steps": 8,
      "speed": 0.95,
      "format": "ogg"
    }
  }'

Voices

voice	Gender	F0	Source
Noa	female	204 Hz	LibriTTS-R 1088 Christabel
Lily	female	211 Hz	LibriTTS-R 6147 Liberty Stump
Daniel	male	128 Hz	LibriTTS-R 6209 deckerteach
Adam	male	112 Hz	LibriTTS-R 8088 Jason Bolestridge

The voice field also accepts the underlying file stem. The files are named after the voice (voices/daniel.json), and the LibriTTS-R reader ids they used to carry still resolve, so a client pinned to "libri_male_6209" keeps landing on Daniel. Neither "Female" nor "Male" names anything in this bundle, so a v2 client sending either gets an error listing the current names — "Female" used to resolve to Maya, which has been withdrawn.

LibriTTS-R is CC BY 4.0 (Google LLC).

Languages

lang accepts en, he, ru, es, de, it. Anything else returns an error listing the supported set rather than mispronouncing the text.

Russian is included by default. RUAccent + RUPhon need torch and transformers, installed from the CPU wheel index (the front end runs on CPU by design), and their ~1 GB of weights is baked into the image so no cold worker downloads them mid-job. That costs ~2 GB of image; --build-arg WITH_RUSSIAN=0 builds the torch-free image without Russian.

Successful output includes:

• audio_base64 — OGG/Opus by default (format: "wav" to keep WAV)
• g2p_s, inference_s, encode_s, total_s — stage timings
• duration_s, rtf, voice, lang

Concurrency is controlled by endpoint env MAX_CONCURRENCY (image default 4, clamp 1–16). GPU synth is serialized with a lock; encode/G2P can overlap via async to_thread.

Build (from repo root)

Models must already be present locally (onnx_slim/, renikud.onnx, voices/), or download first:

python download_models.py

Then:

docker buildx build \
  --platform linux/amd64 \
  -f docker/Dockerfile \
  -t maxme123/blue-v2:latest \
  --push .

Without Russian (no torch, ~2 GB smaller):

docker buildx build \
  --platform linux/amd64 \
  --build-arg WITH_RUSSIAN=0 \
  -f docker/Dockerfile \
  -t maxme123/blue-v2:no-russian \
  --push .

Local load only (no push):

docker buildx build \
  --platform linux/amd64 \
  -f docker/Dockerfile \
  -t maxme123/blue-v2:latest \
  --load .

Run locally (NVIDIA)

docker run --rm --gpus all -p 7860:7860 maxme123/blue-v2:latest

Open http://localhost:7860

RunPod

1. Create a GPU Pod (or custom template).
2. Container image: maxme123/blue-v2:latest
3. Expose HTTP port 7860 (map container port 7860).
4. Ensure the pod has an NVIDIA GPU + NVIDIA Container Toolkit (RunPod default).
5. Start the pod and open the HTTP service URL.

Env knobs (optional):

Variable	Default	Meaning
MAX_CONCURRENCY	4	Concurrent jobs per worker (1–16); set on endpoint without rebuild
BLUE_ONNX_GPU	1	Use ONNX CUDA EP
ORT_NUM_THREADS	4	ORT CPU thread pool
BLUE_PRELOAD	0	Set 1 to load ONNX before first job
BLUE_ONNX_REPO	notmax123/BlueTTS2.5-onnx	Bundle to fetch when onnx_slim/ is incomplete
BLUE_RENIKUD_REPO	notmax123/RenikudPlus	Hebrew G2P weights
BLUE_RU_G2P_DEVICE	CPU	Device for RUAccent/RUPhon

Notes

• Default audio is OGG/Opus (ffmpeg in image). Pass "format":"wav" for WAV.
• Build for linux/amd64 — RunPod GPU hosts are x86_64.
• FlashBoot / idleTimeout are endpoint settings in the RunPod console (not in this image).