Model card: search keywords, voice table, FAQ, links

bc5856f verified about 5 hours ago

14.2 kB

license: apache-2.0
language:
  - en
library_name: onnx
pipeline_tag: text-to-speech
tags:
  - text-to-speech
  - tts
  - speech-synthesis
  - neural-tts
  - offline-tts
  - on-device
  - edge-ai
  - edge
  - offline
  - cpu
  - onnx
  - onnxruntime
  - lightweight
  - small-model
  - flow-matching
  - vocos
  - english
  - raspberry-pi
  - flockspeech
  - conversational
  - conversational-ai
  - expressive-tts
  - nonverbal
  - laughter
  - disfluency
  - voice-assistant
  - game-audio
datasets:
  - amaai-lab/DisfluencySpeech

Goldfinch v0.1: tiny conversational text-to-speech (TTS) that laughs on cue

A small, expressive English text-to-speech model that laughs, breathes and says "uh", offline on any CPU. 44 MB, no GPU, no cloud.

Goldfinch is an 11.4M-parameter flow-matching TTS model from Flock Speech. Where its sibling Junco reads aloud, Goldfinch talks: it keeps fillers such as "uh" and "you know", restarts like "I, I think", and non-speech sounds you place with tags such as [laugh]. It runs faster than real time on an ordinary CPU, with no GPU, network or PyTorch dependency.

Goldfinch is the conversational voice of the Flock Speech family, for chat companions, game characters, conversational AI and voice assistants that should sound casual and human, on-device and offline. Install it with pip install flockspeech.

Flock Speech voices	Style	Model card
Goldfinch	conversational voice that laughs and breathes on cue with `[laugh]` tags	this page
Junco	clear, neutral reading voice	flockspeech/junco


Developed by	Flock Speech
Model type	Non-autoregressive TTS: transformer encoder, duration predictor, conditional flow-matching decoder, Vocos vocoder
Language	English (US)
Voices	1 (conversational style)
Sound tags	`[laugh]`, `[breath]`, `[sigh]`, `[lipsmack]`, `[throat]`, `[noise]`
Output	24 kHz mono audio
Parameters	11.4M acoustic model + 13.5M vocoder
Download	44 MB (ONNX; int8 decoder, fp16 encoder and vocoder). A 100 MB full-precision build is also included
Runtime	ONNX Runtime on CPU (Linux, macOS, Windows; x86-64 and ARM64)
License	Apache-2.0 (vocoder weights MIT)
Version	v0.1, November 2026

Samples

All samples were made with the files in this repository at default settings (8 steps, seed 0), with no editing.

Use	Text	Audio
Laugh	Oh my gosh, [laugh] I can't believe you actually did that.
Chat	Yeah, you know, I used to work there, but, um, not anymore.
Planning	[breath] Okay, so here's the plan for tomorrow. We meet at nine, and, uh, we'll figure out the rest.
Reaction	That's hilarious. [laugh] No, really, it is.
Plain	The weather today is sunny, with a light breeze from the west.

Quick start

The flockspeech Python package (0.2.0 or later) bundles these weights, so nothing is downloaded at run time.

pip install flockspeech
flockspeech "Oh my gosh, [laugh] I can't believe you actually did that." --voice goldfinch -o laugh.wav --play

from flockspeech import Goldfinch

tts = Goldfinch()                    # loads in under a second
audio = tts("Yeah, um, we usually eat dinner around six. [laugh] Late, I know.")   # float32, 24 kHz mono
tts.save("Well, you know, it depends.", "reply.wav")

for chunk in tts.stream(long_text):  # sentence by sentence, for low latency
    play(chunk)

To use the files in this repository directly:

from huggingface_hub import snapshot_download
from flockspeech import Goldfinch

path = snapshot_download("flockspeech/goldfinch", allow_patterns=["onnx/*"])
tts = Goldfinch(model_dir=f"{path}/onnx")

# or the full-precision build (100 MB, about 40% slower on a CPU)
path = snapshot_download("flockspeech/goldfinch", allow_patterns=["onnx-fp32/*"])
tts = Goldfinch(model_dir=f"{path}/onnx-fp32")

Sound tags

Write a tag in square brackets where the sound should go. Each tag is one symbol in the model's input, so the model decides how the sound is shaped and how long it lasts, in the context of the words around it.

Tag	Also accepted	Training examples	Reliability
`[laugh]`	`[laughs]`, `[giggle]`, `[chuckle]`	701	good
`[breath]`	`[inhale]`, `[exhale]`	98	good
`[throat]`	`[clears throat]`, `[cough]`	57	fair
`[lipsmack]`	`[lip smack]`	56	fair
`[noise]`		19	weak
`[sigh]`		16	weak

By default the engine gives each tag 1.4× its predicted length and doubles the short pauses on either side. In our listening comparison this made laughs sound smoother than the raw prediction. These settings are tag_len and tag_pad in onnx/goldfinch.json.

Fillers need no tags: write them as words ("uh", "um", "you know", "I mean").

Options

Parameter	Default	Effect
`speed`	1.0	Speaking rate; 1.2 is 20% faster
`steps`	8	Flow-matching steps; fewer is faster
`seed`	0	Fixes the random variation in delivery, so output is reproducible
`threads`	up to 4	CPU threads for ONNX Runtime

Intended use

Suitable for:

Voice output that should sound casual and human: chat companions, game characters, conversational assistants, podcasts and prototypes of spoken dialogue.
Devices and apps that must work offline or keep text private.
Research on small expressive TTS models and nonverbal sound generation.

Out of scope:

Languages other than English, or speaker voices other than the one provided. Goldfinch has no voice cloning.
Formal reading, such as news, audiobooks or announcements. Use Junco for those.
Emotion control beyond the tags listed above.
Impersonating a real person, or producing audio meant to deceive listeners about who is speaking or whether the speech is synthetic.

Limitations

Weak tags. [sigh] and [noise] had fewer than 20 training examples each, and can come out faint or wrong.
Variable delivery. Goldfinch was trained on spontaneous-sounding speech, so pacing and emphasis vary more than in a read voice. Try another seed if one take sounds off.
No text normalisation. Write numbers, currency, dates and abbreviations in words.
Pronunciation comes from espeak-ng. Rare names and some homographs can be mispronounced.
One voice, one accent. The model learned from a single US English speaker and about 8 hours of audio.
Sentence by sentence. Prosody does not carry across sentences.

Risks and responsible use

Synthetic speech that laughs, breathes and hesitates sounds more human, which makes it easier to mistake for a real person. Goldfinch produces a single fixed voice and cannot clone speakers. We ask that you:

tell listeners when speech is synthetic;
not use the model for fraud, harassment, scams or deceptive impersonation;
review output before you use it in high-stakes settings.

Model details

Architecture

Goldfinch uses the same architecture as Junco:

Text front end: espeak-ng converts text to IPA phonemes (en-us, with stress marks). Each sound tag becomes one extra symbol (6 symbols, ids 177–182). A blank token is inserted between symbols.
Text encoder (3.2M parameters): 3 ConvNeXt pre-net blocks, then 6 transformer layers (width 192, RoPE, SwiGLU, RMSNorm).
Duration predictor (0.3M parameters): a convolutional regressor of log-durations, trained on alignments from Monotonic Alignment Search.
Decoder (7.9M parameters): a flow-matching U-Net hybrid with ConvNeXt blocks at the full frame rate and adaLN transformer blocks at half the frame rate. It uses classifier-free guidance (weight 1.0) and 8 Euler steps with sway sampling, and outputs a 100-band log-mel spectrogram (hop 256, 24 kHz).

The vocoder is Vocos (charactr/vocos-mel-24khz, MIT). In ONNX it predicts the STFT magnitude and phase, and the inverse STFT runs in numpy. onnx/vocoder.onnx is byte-identical to Junco's.

Files

File	Contents	Size
`onnx/encoder.onnx`	Text encoder and duration predictor (fp16 weights, fp32 compute)	7.2 MB
`onnx/decoder-int8.onnx`	One guided flow-matching step (int8 dynamic quantisation)	9.9 MB
`onnx/vocoder.onnx`	Vocos backbone and spectral head (fp16 weights, fp32 compute)	27.1 MB
`onnx/goldfinch.json`	Mel statistics, tag ids and default inference settings	<1 kB
`onnx-fp32/encoder.onnx`	Text encoder and duration predictor, fp32	14.2 MB
`onnx-fp32/decoder.onnx`	One guided flow-matching step, fp32	31.5 MB
`onnx-fp32/vocoder.onnx`	Vocos, fp32	54.2 MB
`onnx-fp32/goldfinch.json`	As above, pointing at the fp32 files	<1 kB
`samples/`	The audio samples above

Training data

DisfluencySpeech (AMAAI Lab, Apache-2.0): one studio speaker re-performing conversations from the Switchboard corpus transcripts, with fillers, restarts and non-speech events kept.

We cleaned it as follows:

Text: filler markup such as {F uh} was kept as words; restarts were kept as spoken; word fragments without a vowel were dropped; each non-speech annotation became one sound tag.
Audio: 60 Hz high-pass filter, resampled to 24 kHz, silence trimmed with a 150 ms margin, loudness normalised to −20 dBFS.
Filtering: clips of 0.8–25 s were kept. We dropped 97 clips that contain speech nobody transcribed, and 50 clips whose length did not fit their text.

The result was 8.3 hours (4,361 clips) for training, and 0.5 hours each for validation and test.

Training


Objective	Conditional flow matching (logit-normal time sampling, σ_min 1e-4), MAS prior loss, duration MSE
Guidance	Condition dropped for 10% of training examples, for classifier-free guidance
Optimiser	AdamW (lr 2e-4 constant, β 0.9/0.98, weight decay 0.01), 1,000 warm-up steps, EMA 0.999
Batching	Dynamic batches of up to 20,000 mel frames
Compute	45,446 steps in about 3.1 hours on one NVIDIA T4 GPU, from scratch
Released weights	EMA weights

A cosine learning-rate schedule trained in parallel did no better, so we released the simpler constant schedule.

Environmental impact. About 6.2 GPU-hours on NVIDIA T4s (70 W TDP), for both runs, or about 0.43 kWh.

FAQ

How do I make it laugh? Write [laugh] where the laugh should go: Goldfinch()("That's hilarious. [laugh] No, really."). The same works for [breath], [throat], [lipsmack], [sigh] and [noise]; see Sound tags.

Do I need to tag "uh" and "um"? No. Write fillers as ordinary words ("uh", "um", "you know", "I mean") and Goldfinch says them naturally.

Does it need a GPU or an internet connection? No. It runs on an ordinary CPU with ONNX Runtime. Once installed, speech is generated fully offline and your text never leaves the machine.

How big is it? 44.2 MB in total: a 17.1 MB acoustic model plus the 27.1 MB Vocos vocoder, shared with Junco. Both voices together take 61.3 MB. A full-precision (fp32) build is also in this repository.

Can it run on a Raspberry Pi or other ARM boards? It runs on 64-bit ARM Linux wherever onnxruntime and espeakng-loader publish wheels, including 64-bit Raspberry Pi OS.

Does it support voice cloning or emotions? No voice cloning: Goldfinch is one English (US) voice. Expressiveness comes from fillers and the sound tags above, not from emotion labels.

Can I use it commercially? The weights are Apache-2.0 and the vocoder is MIT. The phonemiser (espeak-ng, phonemizer) is GPL-3.0 and is installed separately, and the training texts come from Switchboard transcripts; see the License section below.

License

Goldfinch weights and configuration: Apache License 2.0.
Training data: DisfluencySpeech, released by its authors under Apache-2.0. Its texts come from Switchboard transcripts, and the Switchboard corpus itself is distributed by the Linguistic Data Consortium (LDC) under its own licence. The recordings and the released model are new works, but if your use is sensitive to data provenance, review these terms yourself.

Run-time dependency. Text-to-phoneme conversion uses espeak-ng and phonemizer. Both are GPL-3.0 and are not included in this repository. See NOTICE.

Citation

@misc{flockspeech2026goldfinch,
  title        = {Goldfinch: a small conversational flow-matching text-to-speech model with sound tags},
  author       = {{Flock Speech}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/flockspeech/goldfinch}},
  note         = {Version 0.1}
}

Acknowledgements

Goldfinch is trained on DisfluencySpeech from the AMAAI Lab. It builds on ideas from Matcha-TTS, Glow-TTS, F5-TTS, ZipVoice, SupertonicTTS and Vocos. We thank the authors of these works.

flockspeech
/

goldfinch