jin9581 commited on
Commit
ff1dcfd
·
verified ·
1 Parent(s): ea9eb32

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -1
README.md CHANGED
@@ -24,7 +24,15 @@ tags:
24
  <img width="100" alt="resemble-logo-horizontal" src="https://github.com/user-attachments/assets/35cf756b-3506-4943-9c72-c05ddfa4e525" />
25
  </div>
26
 
27
- Chatterbox-Flash is a **block-diffusion zero-shot TTS model** that extends the [Chatterbox-TTS](https://github.com/resemble-ai/chatterbox) pipeline with a parallel masked decoder while preserving streaming generation.
 
 
 
 
 
 
 
 
28
 
29
  The released weights cover the four checkpoints needed at inference time:
30
 
@@ -58,5 +66,31 @@ ta.save("out.wav", wav.unsqueeze(0).cpu(), tts.sr)
58
  - CFG with `w = 1.0`, `pmi_cfg` combination
59
  - FlashInfer paged KV cache + CUDA graph capture
60
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
  # License
62
  MIT — see `LICENSE` in the source repository.
 
24
  <img width="100" alt="resemble-logo-horizontal" src="https://github.com/user-attachments/assets/35cf756b-3506-4943-9c72-c05ddfa4e525" />
25
  </div>
26
 
27
+ Chatterbox-Flash is a **block-diffusion zero-shot TTS model** that extends the [Chatterbox-TTS](https://github.com/resemble-ai/chatterbox) pipeline with a parallel masked decoder while preserving streaming generation. It unmasks multiple speech tokens **in parallel within each block** while keeping **native block-by-block streaming**, delivering autoregressive-class quality at a fraction of the latency.
28
+
29
+ # Streaming & Speed Highlights
30
+ - **Native block-by-block streaming** — emits audio as each block is committed, no full-sequence wait.
31
+ - **~9× real-time** synthesis at the default config (`D = 16`, `α = 0.5`), up to **~13× real-time** at `D = 32`, `α = 0.75`.
32
+ - **Time-to-first-packet from 103 ms**, on par with streaming AR systems.
33
+ - **RTF as low as 0.076** — substantially faster than autoregressive streaming TTS at the same scale.
34
+ - **Early-decoding schedule** adaptively ends denoising early, cutting average steps per block by **~20%** at negligible quality cost.
35
+ - Built on a **FlashInfer paged KV cache + CUDA-graph** inference engine for low per-step overhead.
36
 
37
  The released weights cover the four checkpoints needed at inference time:
38
 
 
66
  - CFG with `w = 1.0`, `pmi_cfg` combination
67
  - FlashInfer paged KV cache + CUDA graph capture
68
 
69
+ # Streaming Efficiency
70
+ Latency and throughput at concurrency 1, measured over 50 utterances. **TTFP** is the wall-clock time from request to the first emitted audio packet; **RTF** (real-time factor) is generation time divided by synthesized audio duration — lower is faster, and `RTF < 1` means faster than real time.
71
+
72
+ | Config (25 Hz, 0.5B) | TTFP (ms) ↓ | RTF ↓ |
73
+ | --- | :---: | :---: |
74
+ | `D = 16`, `α = 0.5` (default) | 118 | 0.107 |
75
+ | `D = 16`, `α = 0.75` | 106 | 0.091 |
76
+ | `D = 24`, `α = 0.5` | 119 | 0.100 |
77
+ | `D = 24`, `α = 0.75` | 105 | 0.084 |
78
+ | `D = 32`, `α = 0.5` | 115 | 0.090 |
79
+ | `D = 32`, `α = 0.75` | **103** | **0.076** |
80
+
81
+ Even on a single concurrent request, Chatterbox-Flash sustains roughly **9× real-time** synthesis at the default setting and **~13× real-time** at `D = 32`, `α = 0.75`, while keeping time-to-first-packet low enough for interactive streaming.
82
+
83
+ # Quality (Seed-TTS test-en)
84
+ Zero-shot TTS quality on the Seed-TTS English benchmark, under the canonical configuration (`D = 16`). **SIM-o** is speaker similarity to the reference (higher is better), **WER** is word error rate from ASR transcription (lower is better), and **UTMOS** is a predicted naturalness score (higher is better). Results are shown for our two main decoding settings against the Chatterbox backbone and ground-truth audio for reference.
85
+
86
+ | System | SIM-o ↑ | WER ↓ | UTMOS ↑ |
87
+ | --- | :---: | :---: | :---: |
88
+ | Ground-truth | 0.734 | 2.14 | 3.52 |
89
+ | Chatterbox (AR backbone) | 0.685 | 2.20 | 4.10 |
90
+ | Chatterbox-Flash (`α = 0`) | **0.704** | **1.96** | **4.09** |
91
+ | Chatterbox-Flash (`α = 0.5`, early decoding) | **0.704** | 2.04 | 4.08 |
92
+
93
+ Converting the autoregressive backbone into a block-diffusion decoder improves both speaker similarity (`0.685 → 0.704`) and intelligibility (`2.20 → 1.96` WER) while keeping naturalness essentially unchanged — all while unlocking parallel, streaming-friendly decoding.
94
+
95
  # License
96
  MIT — see `LICENSE` in the source repository.