# BlueTTS 2.5 ONNX Docker (NVIDIA GPU)

GPU image for **ONNX Runtime CUDA** inference, intended for [RunPod](https://www.runpod.io/) GPU pods.

Image: **`maxme123/blue-v2`**

## What’s inside

- BlueTTS 2.5 ONNX bundle (`onnx_slim/`, from `notmax123/BlueTTS2.5-onnx`)
- `renikud.onnx` — Hebrew G2P from `notmax123/RenikudPlus`, gender-conditioned
- Five named voices in `voices/` — see below
- `onnxruntime-gpu` with `CUDAExecutionProvider`
- **RunPod serverless** handler (`docker/rp_handler.py`) — default
- Gradio UI on port **7860** when `BLUE_SERVE_MODE=gradio`

## Serverless API

```bash
curl -X POST "https://api.runpod.ai/v2/<ENDPOINT_ID>/runsync" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <API_KEY>" \
  -d '{
    "input": {
      "prompt": "Hello from BlueTTS.",
      "voice": "Daniel",
      "lang": "en",
      "steps": 8,
      "speed": 0.95,
      "format": "ogg"
    }
  }'
```

## Voices

| `voice` | Gender | F0 | Source |
|---|---|---|---|
| `Noa` | female | 204 Hz | LibriTTS-R 1088 Christabel |
| `Lily` | female | 211 Hz | LibriTTS-R 6147 Liberty Stump |
| `Daniel` | male | 128 Hz | LibriTTS-R 6209 deckerteach |
| `Adam` | male | 112 Hz | LibriTTS-R 8088 Jason Bolestridge |

The `voice` field also accepts the underlying file stem. The files are named
after the voice (`voices/daniel.json`), and the LibriTTS-R reader ids they used
to carry still resolve, so a client pinned to `"libri_male_6209"` keeps landing
on `Daniel`. Neither `"Female"` nor `"Male"` names anything in this bundle, so a
v2 client sending either gets an error listing the current names — `"Female"`
used to resolve to Maya, which has been withdrawn.

LibriTTS-R is CC BY 4.0 (Google LLC).

## Languages

`lang` accepts `en`, `he`, `ru`, `es`, `de`, `it`. Anything else returns an
error listing the supported set rather than mispronouncing the text.

Russian is included by default. RUAccent + RUPhon need torch and transformers,
installed from the CPU wheel index (the front end runs on CPU by design), and
their ~1 GB of weights is baked into the image so no cold worker downloads them
mid-job. That costs ~2 GB of image; `--build-arg WITH_RUSSIAN=0` builds the
torch-free image without Russian.

Successful `output` includes:
- `audio_base64` — **OGG/Opus by default** (`format: "wav"` to keep WAV)
- `g2p_s`, `inference_s`, `encode_s`, `total_s` — stage timings
- `duration_s`, `rtf`, `voice`, `lang`

Concurrency is controlled by endpoint env **`MAX_CONCURRENCY`** (image default `4`, clamp 1–16). GPU synth is serialized with a lock; encode/G2P can overlap via async `to_thread`.

## Build (from repo root)

Models must already be present locally (`onnx_slim/`, `renikud.onnx`, `voices/`), or download first:

```bash
python download_models.py
```

Then:

```bash
docker buildx build \
  --platform linux/amd64 \
  -f docker/Dockerfile \
  -t maxme123/blue-v2:latest \
  --push .
```

Without Russian (no torch, ~2 GB smaller):

```bash
docker buildx build \
  --platform linux/amd64 \
  --build-arg WITH_RUSSIAN=0 \
  -f docker/Dockerfile \
  -t maxme123/blue-v2:no-russian \
  --push .
```

Local load only (no push):

```bash
docker buildx build \
  --platform linux/amd64 \
  -f docker/Dockerfile \
  -t maxme123/blue-v2:latest \
  --load .
```

## Run locally (NVIDIA)

```bash
docker run --rm --gpus all -p 7860:7860 maxme123/blue-v2:latest
```

Open http://localhost:7860

## RunPod

1. Create a **GPU Pod** (or custom template).
2. Container image: `maxme123/blue-v2:latest`
3. Expose HTTP port **7860** (map container port 7860).
4. Ensure the pod has an NVIDIA GPU + NVIDIA Container Toolkit (RunPod default).
5. Start the pod and open the HTTP service URL.

Env knobs (optional):

| Variable | Default | Meaning |
|---|---|---|
| `MAX_CONCURRENCY` | `4` | Concurrent jobs per worker (1–16); set on endpoint without rebuild |
| `BLUE_ONNX_GPU` | `1` | Use ONNX CUDA EP |
| `ORT_NUM_THREADS` | `4` | ORT CPU thread pool |
| `BLUE_PRELOAD` | `0` | Set `1` to load ONNX before first job |
| `BLUE_ONNX_REPO` | `notmax123/BlueTTS2.5-onnx` | Bundle to fetch when `onnx_slim/` is incomplete |
| `BLUE_RENIKUD_REPO` | `notmax123/RenikudPlus` | Hebrew G2P weights |
| `BLUE_RU_G2P_DEVICE` | `CPU` | Device for RUAccent/RUPhon |

## Notes

- Default audio is **OGG/Opus** (ffmpeg in image). Pass `"format":"wav"` for WAV.
- Build for **`linux/amd64`** — RunPod GPU hosts are x86_64.
- FlashBoot / `idleTimeout` are endpoint settings in the RunPod console (not in this image).