File size: 6,221 Bytes

---
license: mit
language:
  - en
  - es
pipeline_tag: text-to-speech
tags:
  - piper
  - vits
  - text-to-speech
  - synaptics
  - torq
  - npu
---

# Piper TTS for Synaptics Torq (SL2619)

Piper (VITS) text-to-speech **split across CPU and NPU** for the Synaptics SL2619
board, in three voices:

| Voice | Language | Sample rate | Speakers | Assets |
|---|---|---|---|---|
| `en_US-libritts_r-medium` (default) | English (US) | 22.05 kHz | 904 | top level |
| `en_US-lessac-low` | English (US) | 16 kHz | 1 | `en_US-lessac-low/` |
| `es_MX-ald-medium` | Spanish (Mexico) | 22.05 kHz | 1 | `es_MX-ald-medium/` |

The VITS graph is cut where the per-phoneme durations are ceiled and summed —
the point at which the output length becomes exact:

```
text --[espeak]--> phoneme ids --> [partA]  (CPU, onnxruntime)  --> z [1,192,F] (+ g [1,512,1])
                                           z (,g) --> [partB]   (NPU, bf16 vmfb) --> audio [F*256]
```

partA holds 85% of the *nodes* (small shape/attention ops) but partB — the
HiFi-GAN vocoder — holds most of the *time* and is pure convolution, which is
what the NPU accelerates. Because partA yields the exact frame count `F`, the
right static vocoder window is known before the vocoder runs. `g` is the speaker
embedding: the single-speaker voices have none, so their vocoder takes `z` alone.

## Performance (SL2619 board)

A 3-sentence sample, text in to all audio synthesized (phonemization included,
model load excluded), median of 5 runs:

| Voice | Audio | CPU only | CPU + NPU | Speedup | First audio (CPU / CPU + NPU) |
|---|---|---|---|---|---|
| `en_US-libritts_r-medium` | 7.70 s | 5.55 s (1.39× RT) | **2.83 s (2.72× RT)** | **1.96×** | 0.95 s / **0.77 s** |
| `en_US-lessac-low` | 9.82 s | 5.25 s (1.87× RT) | **2.51 s (3.92× RT)** | **2.09×** | 0.89 s / **0.68 s** |
| `es_MX-ald-medium` | 12.63 s | 8.66 s (1.46× RT) | **4.07 s (3.10× RT)** | **2.13×** | 1.81 s / **1.38 s** |

"CPU only" is the full Piper voice in onnxruntime (2 threads), one sentence at a
time; "CPU + NPU" is partA on the CPU pipelined with the vocoder on the NPU. The
English and Spanish samples are the same three sentences, translated.

The vocoder alone on the NPU, per window:

| window | libritts / Spanish (22.05 kHz) | lessac-low (16 kHz) |
|---|---|---|
| 1 s | 170 ms (5.9× RT) | 126 ms (7.8× RT) |
| 1.5 s | — | 186 ms (8.1× RT) |
| 2 s | 335 ms (6.0× RT) | 249 ms (8.0× RT) |
| 3 s | — | 365 ms (8.2× RT) |
| 4 s | 688 ms (5.8× RT) | 497 ms (8.0× RT) |
| 5 s | — | 628 ms (8.0× RT) |
| 6 s | 1032 ms (5.8× RT) | 749 ms (8.0× RT) |
| 8 s | 1356 ms (5.9× RT) | 1027 ms (7.8× RT) |

The CPU+NPU pipeline overlaps the halves: the CPU encodes sentence *n+1* while the
NPU vocodes sentence *n* and the speaker plays sentence *n-1*.

Accuracy: each bf16 NPU vocoder window matches the fp32 onnxruntime vocoder on the
same latent at 37.3–40.7 dB SNR (correlation ≥ 0.9999).

## Contents

Per voice (English at the top level, the others under their voice-named folder):

| path | what it is |
|---|---|
| `onnx/partA.onnx` | Text encoder + duration predictor. Runs on the CPU under onnxruntime. |
| `vmfb/partB_static_<N>s.vmfb` | The vocoder compiled for the Torq NPU (NSS-only, bf16), one per window: 1, 2, 4, 6, 8 s (lessac-low adds 1.5, 3, 5 s). |
| `voice/<voice>.onnx.json` | Voice config: phoneme→id map, sample rate, espeak language. |
| `onnx/partB_static_<N>s{.bf16io,}.onnx` | The statically shaped bf16-I/O vocoder each VMFB was compiled from. |

Shared, and English-only extras:

| path | what it is |
|---|---|
| `espeak/phonemizerd` | Small resident espeak-ng phonemizer daemon (aarch64) mirroring libpiper's phonemization. |
| `espeak/espeak-ng-data.tar.gz` | espeak-ng dictionaries; one copy covers every language. |
| `onnx/en_US-libritts_r-medium.onnx` | The original monolithic Piper voice, for reference. |
| `tflite/partB_static_4s.{int8,int16x8}.tflite` | Quantized vocoder conversions (int8 was slower than bf16 on this target; kept for reference). |

The vocoder ships as one VMFB per window because the NPU model is statically
shaped: 1, 2, 4, 6 and 8 seconds of audio, plus 1.5, 3 and 5 s for `en_US-lessac-low`.
Per sentence, pick the smallest window that fits, edge-pad the latent up to it, and
trim the output back to `F × 256` samples. lessac-low's finer windows cut padding
from 26% to 16% of vocoder frames (NPU vocoder time −12.5% on a 34-sentence mix;
1–3% end to end, since that voice is CPU-bound once the vocoder is on the NPU).

## How these were built

Every voice is produced by one command in
[torq-tools](https://github.com/synaptics-torq/torq-tools):

```sh
torq-export-model piper -v es_MX-ald-medium     # any rhasspy/piper-voices key
```

It downloads the voice from `rhasspy/piper-voices` (pinned to `v1.0.0`), splits
it, pins the five vocoder windows (sized from the voice's own sample rate),
converts them to bf16 with bf16 I/O, and compiles each NSS-only
(`--torq-disable-css --torq-disable-host`). The VMFBs here were compiled with the
Torq compiler including the HiFi-GAN vocoder fixes (not yet on main; these builds are about 30% faster on the NPU than the first
release of this repo, with bit-identical output).

## Usage

These files are consumed by the `piper_tts` demo in
[synaptics-torq/torq-examples](https://github.com/synaptics-torq/torq-examples),
which writes a `.wav` and plays it on the board's speaker:

```sh
cd piper_tts
pip install -r requirements.txt
cd .. && python setup_demos.py piper_tts
cd piper_tts
python src/infer.py --text "Hello from the Synaptics board."
python src/infer.py --voice en_US-lessac-low --interactive
python src/infer.py --voice es_MX-ald-medium --text "Buenos días."
```

Only the requested voice is downloaded. The demo reads each window's frame width
from the VMFB signature and each voice's sample rate from its config, so
recompiling with a different set of windows needs no code change.

## Licensing

The Piper voices and models are MIT. `espeak/phonemizerd` links espeak-ng
(**GPLv3**) statically; its source ships with the demo at
`piper_tts/piper_core/phonemizerd.c` in torq-examples, with the build command in
its header comment.