| # BlueTTS 2.5 ONNX Docker (NVIDIA GPU) |
|
|
| GPU image for **ONNX Runtime CUDA** inference, intended for [RunPod](https://www.runpod.io/) GPU pods. |
|
|
| Image: **`maxme123/blue-v2`** |
|
|
| ## What’s inside |
|
|
| - BlueTTS 2.5 ONNX bundle (`onnx_slim/`, from `notmax123/BlueTTS2.5-onnx`) |
| - `renikud.onnx` — Hebrew G2P from `notmax123/RenikudPlus`, gender-conditioned |
| - Five named voices in `voices/` — see below |
| - `onnxruntime-gpu` with `CUDAExecutionProvider` |
| - **RunPod serverless** handler (`docker/rp_handler.py`) — default |
| - Gradio UI on port **7860** when `BLUE_SERVE_MODE=gradio` |
|
|
| ## Serverless API |
|
|
| ```bash |
| curl -X POST "https://api.runpod.ai/v2/<ENDPOINT_ID>/runsync" \ |
| -H "Content-Type: application/json" \ |
| -H "Authorization: Bearer <API_KEY>" \ |
| -d '{ |
| "input": { |
| "prompt": "Hello from BlueTTS.", |
| "voice": "Daniel", |
| "lang": "en", |
| "steps": 8, |
| "speed": 0.95, |
| "format": "ogg" |
| } |
| }' |
| ``` |
|
|
| ## Voices |
|
|
| | `voice` | Gender | F0 | Source | |
| |---|---|---|---| |
| | `Noa` | female | 204 Hz | LibriTTS-R 1088 Christabel | |
| | `Lily` | female | 211 Hz | LibriTTS-R 6147 Liberty Stump | |
| | `Daniel` | male | 128 Hz | LibriTTS-R 6209 deckerteach | |
| | `Adam` | male | 112 Hz | LibriTTS-R 8088 Jason Bolestridge | |
|
|
| The `voice` field also accepts the underlying file stem. The files are named |
| after the voice (`voices/daniel.json`), and the LibriTTS-R reader ids they used |
| to carry still resolve, so a client pinned to `"libri_male_6209"` keeps landing |
| on `Daniel`. Neither `"Female"` nor `"Male"` names anything in this bundle, so a |
| v2 client sending either gets an error listing the current names — `"Female"` |
| used to resolve to Maya, which has been withdrawn. |
|
|
| LibriTTS-R is CC BY 4.0 (Google LLC). |
|
|
| ## Languages |
|
|
| `lang` accepts `en`, `he`, `ru`, `es`, `de`, `it`. Anything else returns an |
| error listing the supported set rather than mispronouncing the text. |
|
|
| Russian is included by default. RUAccent + RUPhon need torch and transformers, |
| installed from the CPU wheel index (the front end runs on CPU by design), and |
| their ~1 GB of weights is baked into the image so no cold worker downloads them |
| mid-job. That costs ~2 GB of image; `--build-arg WITH_RUSSIAN=0` builds the |
| torch-free image without Russian. |
|
|
| Successful `output` includes: |
| - `audio_base64` — **OGG/Opus by default** (`format: "wav"` to keep WAV) |
| - `g2p_s`, `inference_s`, `encode_s`, `total_s` — stage timings |
| - `duration_s`, `rtf`, `voice`, `lang` |
|
|
| Concurrency is controlled by endpoint env **`MAX_CONCURRENCY`** (image default `4`, clamp 1–16). GPU synth is serialized with a lock; encode/G2P can overlap via async `to_thread`. |
| |
| ## Build (from repo root) |
| |
| Models must already be present locally (`onnx_slim/`, `renikud.onnx`, `voices/`), or download first: |
| |
| ```bash |
| python download_models.py |
| ``` |
| |
| Then: |
| |
| ```bash |
| docker buildx build \ |
| --platform linux/amd64 \ |
| -f docker/Dockerfile \ |
| -t maxme123/blue-v2:latest \ |
| --push . |
| ``` |
| |
| Without Russian (no torch, ~2 GB smaller): |
| |
| ```bash |
| docker buildx build \ |
| --platform linux/amd64 \ |
| --build-arg WITH_RUSSIAN=0 \ |
| -f docker/Dockerfile \ |
| -t maxme123/blue-v2:no-russian \ |
| --push . |
| ``` |
| |
| Local load only (no push): |
| |
| ```bash |
| docker buildx build \ |
| --platform linux/amd64 \ |
| -f docker/Dockerfile \ |
| -t maxme123/blue-v2:latest \ |
| --load . |
| ``` |
| |
| ## Run locally (NVIDIA) |
| |
| ```bash |
| docker run --rm --gpus all -p 7860:7860 maxme123/blue-v2:latest |
| ``` |
| |
| Open http://localhost:7860 |
| |
| ## RunPod |
| |
| 1. Create a **GPU Pod** (or custom template). |
| 2. Container image: `maxme123/blue-v2:latest` |
| 3. Expose HTTP port **7860** (map container port 7860). |
| 4. Ensure the pod has an NVIDIA GPU + NVIDIA Container Toolkit (RunPod default). |
| 5. Start the pod and open the HTTP service URL. |
| |
| Env knobs (optional): |
| |
| | Variable | Default | Meaning | |
| |---|---|---| |
| | `MAX_CONCURRENCY` | `4` | Concurrent jobs per worker (1–16); set on endpoint without rebuild | |
| | `BLUE_ONNX_GPU` | `1` | Use ONNX CUDA EP | |
| | `ORT_NUM_THREADS` | `4` | ORT CPU thread pool | |
| | `BLUE_PRELOAD` | `0` | Set `1` to load ONNX before first job | |
| | `BLUE_ONNX_REPO` | `notmax123/BlueTTS2.5-onnx` | Bundle to fetch when `onnx_slim/` is incomplete | |
| | `BLUE_RENIKUD_REPO` | `notmax123/RenikudPlus` | Hebrew G2P weights | |
| | `BLUE_RU_G2P_DEVICE` | `CPU` | Device for RUAccent/RUPhon | |
| |
| ## Notes |
| |
| - Default audio is **OGG/Opus** (ffmpeg in image). Pass `"format":"wav"` for WAV. |
| - Build for **`linux/amd64`** — RunPod GPU hosts are x86_64. |
| - FlashBoot / `idleTimeout` are endpoint settings in the RunPod console (not in this image). |
| |
