Text-to-Speech
ONNX
English
tts
speech-synthesis
neural-tts
offline-tts
on-device
edge-ai
edge
offline
cpu
onnxruntime
lightweight
small-model
flow-matching
vocos
english
raspberry-pi
flockspeech
conversational
conversational-ai
expressive-tts
nonverbal
laughter
disfluency
voice-assistant
game-audio
Model card: search keywords, voice table, FAQ, links
Browse files
README.md
CHANGED
|
@@ -8,28 +8,53 @@ tags:
|
|
| 8 |
- text-to-speech
|
| 9 |
- tts
|
| 10 |
- speech-synthesis
|
| 11 |
-
-
|
| 12 |
-
-
|
| 13 |
-
- laughter
|
| 14 |
-
- onnx
|
| 15 |
-
- flow-matching
|
| 16 |
- on-device
|
|
|
|
| 17 |
- edge
|
| 18 |
- offline
|
| 19 |
- cpu
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
datasets:
|
| 21 |
- amaai-lab/DisfluencySpeech
|
| 22 |
---
|
| 23 |
|
| 24 |
-
# Goldfinch v0.1
|
| 25 |
|
| 26 |
-
**A small
|
|
|
|
| 27 |
|
| 28 |
Goldfinch is an 11.4M-parameter flow-matching TTS model from [Flock Speech](https://huggingface.co/flockspeech). Where
|
| 29 |
its sibling [Junco](https://huggingface.co/flockspeech/junco) reads aloud, Goldfinch talks: it keeps fillers such as
|
| 30 |
"uh" and "you know", restarts like "I, I think", and non-speech sounds you place with tags such as `[laugh]`. It runs
|
| 31 |
faster than real time on an ordinary CPU, with no GPU, network or PyTorch dependency.
|
| 32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
| | |
|
| 34 |
|---|---|
|
| 35 |
| **Developed by** | Flock Speech |
|
|
@@ -217,6 +242,35 @@ A cosine learning-rate schedule trained in parallel did no better, so we release
|
|
| 217 |
|
| 218 |
**Environmental impact.** About 6.2 GPU-hours on NVIDIA T4s (70 W TDP), for both runs, or about 0.43 kWh.
|
| 219 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 220 |
## License
|
| 221 |
|
| 222 |
- **Goldfinch weights and configuration:** [Apache License 2.0](LICENSE).
|
|
@@ -241,6 +295,12 @@ included in this repository. See [NOTICE](NOTICE).
|
|
| 241 |
}
|
| 242 |
```
|
| 243 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 244 |
## Acknowledgements
|
| 245 |
|
| 246 |
Goldfinch is trained on DisfluencySpeech from the AMAAI Lab. It builds on ideas from
|
|
|
|
| 8 |
- text-to-speech
|
| 9 |
- tts
|
| 10 |
- speech-synthesis
|
| 11 |
+
- neural-tts
|
| 12 |
+
- offline-tts
|
|
|
|
|
|
|
|
|
|
| 13 |
- on-device
|
| 14 |
+
- edge-ai
|
| 15 |
- edge
|
| 16 |
- offline
|
| 17 |
- cpu
|
| 18 |
+
- onnx
|
| 19 |
+
- onnxruntime
|
| 20 |
+
- lightweight
|
| 21 |
+
- small-model
|
| 22 |
+
- flow-matching
|
| 23 |
+
- vocos
|
| 24 |
+
- english
|
| 25 |
+
- raspberry-pi
|
| 26 |
+
- flockspeech
|
| 27 |
+
- conversational
|
| 28 |
+
- conversational-ai
|
| 29 |
+
- expressive-tts
|
| 30 |
+
- nonverbal
|
| 31 |
+
- laughter
|
| 32 |
+
- disfluency
|
| 33 |
+
- voice-assistant
|
| 34 |
+
- game-audio
|
| 35 |
datasets:
|
| 36 |
- amaai-lab/DisfluencySpeech
|
| 37 |
---
|
| 38 |
|
| 39 |
+
# Goldfinch v0.1: tiny conversational text-to-speech (TTS) that laughs on cue
|
| 40 |
|
| 41 |
+
**A small, expressive English text-to-speech model that laughs, breathes and says "uh", offline on any CPU. 44 MB, no
|
| 42 |
+
GPU, no cloud.**
|
| 43 |
|
| 44 |
Goldfinch is an 11.4M-parameter flow-matching TTS model from [Flock Speech](https://huggingface.co/flockspeech). Where
|
| 45 |
its sibling [Junco](https://huggingface.co/flockspeech/junco) reads aloud, Goldfinch talks: it keeps fillers such as
|
| 46 |
"uh" and "you know", restarts like "I, I think", and non-speech sounds you place with tags such as `[laugh]`. It runs
|
| 47 |
faster than real time on an ordinary CPU, with no GPU, network or PyTorch dependency.
|
| 48 |
|
| 49 |
+
Goldfinch is the **conversational voice** of the Flock Speech family, for chat companions, game characters,
|
| 50 |
+
conversational AI and voice assistants that should sound casual and human, on-device and offline. Install it with
|
| 51 |
+
`pip install flockspeech`.
|
| 52 |
+
|
| 53 |
+
| Flock Speech voices | Style | Model card |
|
| 54 |
+
|---|---|---|
|
| 55 |
+
| **Goldfinch** | conversational voice that laughs and breathes on cue with `[laugh]` tags | this page |
|
| 56 |
+
| **Junco** | clear, neutral reading voice | [flockspeech/junco](https://huggingface.co/flockspeech/junco) |
|
| 57 |
+
|
| 58 |
| | |
|
| 59 |
|---|---|
|
| 60 |
| **Developed by** | Flock Speech |
|
|
|
|
| 242 |
|
| 243 |
**Environmental impact.** About 6.2 GPU-hours on NVIDIA T4s (70 W TDP), for both runs, or about 0.43 kWh.
|
| 244 |
|
| 245 |
+
## FAQ
|
| 246 |
+
|
| 247 |
+
**How do I make it laugh?**
|
| 248 |
+
Write `[laugh]` where the laugh should go: `Goldfinch()("That's hilarious. [laugh] No, really.")`. The same works for
|
| 249 |
+
`[breath]`, `[throat]`, `[lipsmack]`, `[sigh]` and `[noise]`; see [Sound tags](#sound-tags).
|
| 250 |
+
|
| 251 |
+
**Do I need to tag "uh" and "um"?**
|
| 252 |
+
No. Write fillers as ordinary words ("uh", "um", "you know", "I mean") and Goldfinch says them naturally.
|
| 253 |
+
|
| 254 |
+
**Does it need a GPU or an internet connection?**
|
| 255 |
+
No. It runs on an ordinary CPU with ONNX Runtime. Once installed, speech is generated fully offline and your text
|
| 256 |
+
never leaves the machine.
|
| 257 |
+
|
| 258 |
+
**How big is it?**
|
| 259 |
+
44.2 MB in total: a 17.1 MB acoustic model plus the 27.1 MB Vocos vocoder, shared with Junco. Both voices together
|
| 260 |
+
take 61.3 MB. A full-precision (fp32) build is also in this repository.
|
| 261 |
+
|
| 262 |
+
**Can it run on a Raspberry Pi or other ARM boards?**
|
| 263 |
+
It runs on 64-bit ARM Linux wherever `onnxruntime` and `espeakng-loader` publish wheels, including 64-bit Raspberry
|
| 264 |
+
Pi OS.
|
| 265 |
+
|
| 266 |
+
**Does it support voice cloning or emotions?**
|
| 267 |
+
No voice cloning: Goldfinch is one English (US) voice. Expressiveness comes from fillers and the sound tags above,
|
| 268 |
+
not from emotion labels.
|
| 269 |
+
|
| 270 |
+
**Can I use it commercially?**
|
| 271 |
+
The weights are Apache-2.0 and the vocoder is MIT. The phonemiser (espeak-ng, phonemizer) is GPL-3.0 and is installed
|
| 272 |
+
separately, and the training texts come from Switchboard transcripts; see the License section below.
|
| 273 |
+
|
| 274 |
## License
|
| 275 |
|
| 276 |
- **Goldfinch weights and configuration:** [Apache License 2.0](LICENSE).
|
|
|
|
| 295 |
}
|
| 296 |
```
|
| 297 |
|
| 298 |
+
## Links
|
| 299 |
+
|
| 300 |
+
- Python package: [pypi.org/project/flockspeech](https://pypi.org/project/flockspeech/)
|
| 301 |
+
- Source code: [github.com/flockspeech/flockspeech](https://github.com/flockspeech/flockspeech)
|
| 302 |
+
- Reading voice: [flockspeech/junco](https://huggingface.co/flockspeech/junco)
|
| 303 |
+
|
| 304 |
## Acknowledgements
|
| 305 |
|
| 306 |
Goldfinch is trained on DisfluencySpeech from the AMAAI Lab. It builds on ideas from
|