rtx 3080 benchmarks and updated 3060 numbers
Browse files
README.md
CHANGED
|
@@ -24,7 +24,8 @@ GGUF conversions of [BreezeBlue/Breeze-TTS-2](https://huggingface.co/BreezeBlue/
|
|
| 24 |
[**Breeze-TTS-2.cpp**](https://github.com/HoppouAI/Breeze-TTS-2.cpp), a C++ reimplementation running on
|
| 25 |
ggml with a Vulkan backend, so it works on NVIDIA, AMD and Intel GPUs and falls back to CPU.
|
| 26 |
|
| 27 |
-
Bilingual English and Mandarin, 24 kHz,
|
|
|
|
| 28 |
|
| 29 |
These files will not load in llama.cpp. They need the Breeze-TTS-2.cpp runtime, which implements all
|
| 30 |
four stages of the model: the T5Gemma2 text encoder, the Qwen3 backbone, the 15 step depth decoder and
|
|
@@ -60,6 +61,28 @@ narration is not, and the failure is gradual rather than obvious, so it is easy
|
|
| 60 |
|
| 61 |
Treat them as experimental. If in doubt, use `q8_0` or `q4_k`.
|
| 62 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
## Usage
|
| 64 |
|
| 65 |
```bash
|
|
|
|
| 24 |
[**Breeze-TTS-2.cpp**](https://github.com/HoppouAI/Breeze-TTS-2.cpp), a C++ reimplementation running on
|
| 25 |
ggml with a Vulkan backend, so it works on NVIDIA, AMD and Intel GPUs and falls back to CPU.
|
| 26 |
|
| 27 |
+
Bilingual English and Mandarin, 24 kHz, about 2x realtime at Q8_0 on an RTX 3060 and about 3.2x on
|
| 28 |
+
an RTX 3080, with first audio in under 150 ms.
|
| 29 |
|
| 30 |
These files will not load in llama.cpp. They need the Breeze-TTS-2.cpp runtime, which implements all
|
| 31 |
four stages of the model: the T5Gemma2 text encoder, the Qwen3 backbone, the 15 step depth decoder and
|
|
|
|
| 61 |
|
| 62 |
Treat them as experimental. If in doubt, use `q8_0` or `q4_k`.
|
| 63 |
|
| 64 |
+
## Performance
|
| 65 |
+
|
| 66 |
+
Real time factor for one stream on Vulkan: seconds of audio per second of wall clock, so anything
|
| 67 |
+
above 1x is faster than playback. `cfg_scale` above 1 turns on guidance, which vocal tags and voice
|
| 68 |
+
design often need.
|
| 69 |
+
|
| 70 |
+
| File | RTX 3060, `cfg_scale` 1 | RTX 3060, `cfg_scale` 3 | RTX 3080, `cfg_scale` 1 | RTX 3080, `cfg_scale` 3 |
|
| 71 |
+
| --- | --- | --- | --- | --- |
|
| 72 |
+
| `q4_k` | 2.41x | 1.74x | 3.50x | 2.81x |
|
| 73 |
+
| `q6_k` | | | 3.67x | 2.87x |
|
| 74 |
+
| `q8_0-dd4` | 2.32x | 1.69x | 3.72x | 2.83x |
|
| 75 |
+
| `q8_0` | 1.99x | 1.58x | 3.16x | 2.71x |
|
| 76 |
+
| `f16` | 1.42x | | 2.71x | 2.32x |
|
| 77 |
+
|
| 78 |
+
The RTX 3060 has its tensor core limit unlocked, the RTX 3080 is a stock card. Both are 12 GB.
|
| 79 |
+
On the 3060 the smaller files are clearly faster, on the 3080 the gap mostly closes.
|
| 80 |
+
|
| 81 |
+
The server can also generate several streams at once with `--parallel`. On the RTX 3080, five streams
|
| 82 |
+
at `q8_0` each ran at 1.46x to 1.68x, and three with guidance at about 1.5x, all with no gaps. The
|
| 83 |
+
numbers for every model and both cards are in
|
| 84 |
+
[docs/performance.md](https://github.com/HoppouAI/Breeze-TTS-2.cpp/blob/main/docs/performance.md).
|
| 85 |
+
|
| 86 |
## Usage
|
| 87 |
|
| 88 |
```bash
|