Silencio

Small zero-shot text-to-speech that speaks in any voice.

Silencio turns English text into 16 kHz speech in a voice it has never heard before. A few seconds of reference audio are enough. All 33.6M parameters fit in one 134 MB model.safetensors. It renders audio in 120 ms chunks and runs about 9x faster than real time on one CPU thread of an Apple M4.

The same weights also do voice conversion: give it speech instead of text and it says the same words in another voice.

Silencio is built on two open models. The text front-end comes from Paradee-8M v1.1 by Sahil Mahendrakar, itself distilled from Kokoro-82M by hexgrad. The voice renderer and vocoder come from MeanVC2 by ASLP-lab. A new 3.4M-parameter content adapter connects them, so text goes straight into the renderer with no intermediate audio.

Model details

• Model type: zero-shot text-to-speech. A text encoder predicts phoneme durations, pitch and phonetic features. A ConvNeXt adapter turns them into content tokens at 25 Hz. A mean-flow DiT renders a mel spectrogram from those tokens and a 256-d voice embedding in 3 steps. Vocos then makes the waveform.
• Parameters: 33.6M (text encoder 4.2M, adapter 3.4M, renderer 17.7M, vocoder 8.3M)
• Weights files: model.safetensors, 134 MB (fp32), and model.int8.safetensors, 34 MB (int8, practically the same quality)
• Language: English (American pronunciation)
• Voices: zero-shot, from a 5–15 s reference clip. 8 ready-made voices are included.
• Output: mono, 16 kHz
• License: Apache 2.0

How it fits together

text ─► misaki G2P ─► text encoder ─► content adapter ─► DiT renderer ─► Vocos ─► 16 kHz audio
        (phonemes)    durations, F0,   content tokens     mean flow,        ▲
                      features @40 Hz  @25 Hz             3 steps           │
                                                            ▲               │
                                          voice embedding ──┘ (256-d)       │
speech ─► Fast-U2++ streaming ASR encoder ─► content tokens ─┘  (voice conversion)

The text encoder predicts how long each phoneme lasts and how the pitch moves. The renderer only decides how it sounds in the chosen voice. So timing and intonation come from the text, and timbre comes from the voice embedding.

Quickstart

Try it in Colab

Open the demo notebook and run all cells. It opens a Gradio demo with text to speech, voice cloning from the microphone or a file, and voice conversion.

Install

hf download BlueRoseLabs/Silencio --local-dir Silencio
cd Silencio
pip install -r requirements.txt
pip install --no-deps --ignore-requires-python misaki==0.9.4   # G2P; declares python<3.13 but runs on 3.13

Python

from silencio import SilencioTTS

tts = SilencioTTS.from_pretrained("BlueRoseLabs/Silencio")   # or the local folder
tts.save("speech.wav", "Hello from Silencio.", voice="librispeech_1320")

# Streaming: one chunk per sentence, float32 at 16 kHz
for pcm in tts.stream("Speech can start playing right away. The next sentence follows.", voice="librispeech_2094"):
    ...

For the 34 MB int8 weights, pass quant="int8" to from_pretrained. They are dequantized on load, so speed and memory use are the same as fp32.

generate returns a numpy array, and speed (default 1.0) changes the speaking rate. On first use misaki downloads spaCy's small English model (about 13 MB).

Any voice from a short clip

import soundfile as sf
from silencio.enroll import enroll        # pip install s3prl

voice = enroll(*sf.read("reference.wav"))  # 256-d embedding; save it with torch.save and reuse it
tts.save("cloned.wav", "This is my voice, read by Silencio.", voice=voice)

The voice encoder is WavLM-Large with ECAPA-TDNN, the same one MeanVC2 was trained with. It is large (about 1.3 GB) and only needed to add new voices, so it is not part of model.safetensors. It is downloaded on first use. A saved embedding needs nothing extra.

Voice conversion

wav, sr = sf.read("source.wav")
sf.write("converted.wav", tts.convert(wav, sr, voice="librispeech_2094"), 16000)

For speech input, the content tokens come from MeanVC2's streaming Fast-U2++ encoder. It is downloaded from ASLP-lab/MeanVC2 on first use and runs on CPU.

Voices

voices.safetensors holds 8 voices enrolled from LibriSpeech test-clean readers. A reader's voice is called librispeech_<speaker id>. The reference clip used for each one is included as ref_<id>.wav.

Voice	Reference clip
librispeech_1320 (default)	ref_1320.wav
librispeech_2094	ref_2094.wav
librispeech_260	ref_260.wav
librispeech_3575	ref_3575.wav
librispeech_5639	ref_5639.wav
librispeech_6930	ref_6930.wav
librispeech_7127	ref_7127.wav
librispeech_7729	ref_7729.wav

Samples

Five held-out sentences, read by Silencio in three voices. The last column is Paradee-8M v1.1, whose text encoder Silencio uses, in its single voice, for comparison. The sentences are in samples/sentences.txt.

#	librispeech_1320	librispeech_2094	librispeech_260	Paradee-8M v1.1
1	listen	listen	listen	listen
2	listen	listen	listen	listen
3	listen	listen	listen	listen
4	listen	listen	listen	listen
5	listen	listen	listen	listen

Voice conversion: source (LibriSpeech 1320) → converted to librispeech_2094. Whisper transcribes both identically, and the speaker similarity to the target voice is 0.83.

Results

All numbers are on Paradee's 200 held-out WikiText sentences, scored the way the Paradee card scores them. Silencio speaks them in its 8 voices in turn.

Model (voice)	Params	Weights	UTMOS	WER	Speaker similarity	Speed, T4	Speed, 1 CPU thread
Silencio (8 zero-shot voices)	33.6M	134 MB	4.15	7.0%	0.72	3.7x	8.8x (M4) · 1.6x (Colab)
Silencio int8	33.6M	34 MB	4.12	7.2%	0.72	4.0x	1.5x (Colab)
Paradee-8M v1.1 (af_heart, single voice), PyTorch	7.9M	8.5 MB (int8)	4.41	6.3%	–	–	–

• UTMOS is a neural network trained on human ratings. It predicts how natural a clip sounds, on a scale from 1 to 5 (SpeechMOS utmos22_strong).
• WER is the share of words that Whisper (base) transcribes wrongly.
• Speaker similarity is the cosine similarity between the voice of the output and the voice of the reference clip, measured with MeanVC2's WavLM/ECAPA encoder. 1.0 is the same embedding.
• Speed is how many times faster than real time the model runs, counting text to waveform, in PyTorch fp32. CPU speed is measured with one thread, on an Apple M4 (one two-sentence paragraph) and on the Colab host (10 held-out sentences).

Paradee was scored by the same script, in PyTorch, which is why its WER differs a little from the 5.9% on its own card.

Paradee speaks one studio voice at 24 kHz. Silencio speaks voices taken from audiobook recordings, and it does so at 16 kHz. Its UTMOS therefore also reflects the recording quality of the reference voices, not only the model.

How it was made

Only the adapter was trained. Everything else is reused frozen from the two base models.

1. Content space. MeanVC2's renderer takes content tokens from a streaming ASR encoder (256-d at 25 Hz) and a speaker embedding. Content tokens carry what is said and how, but not who says it.
2. Targets. Paradee read 6,000 WikiText sentences at speeds 0.9, 1.0 and 1.1 (about 6 hours). For each one, Paradee's internal text-side features were kept, and MeanVC2's ASR encoder was run on Paradee's audio to get the content tokens that audio would produce.
3. Adapter. A ConvNeXt network (4 blocks at 40 Hz, resampling to 25 Hz, then 4 more blocks) learns to map the text features, pitch and loudness to those content tokens. It uses an L1 plus cosine loss. Training took 15,000 steps at batch 32, about 22 minutes on one T4.
4. One model. The text encoder, adapter, renderer (EMA weights) and vocoder were put into a single model.safetensors. Weight norm was folded into the weights, and the Vocos TorchScript was converted to plain PyTorch. The output of the merged model matches the original chained pipeline to within 7e-4.

In speech-to-speech mode the original ASR encoder takes the adapter's place, so voice conversion works exactly as in MeanVC2.

Files

File	What it is
model.safetensors	The whole model: text encoder, content adapter, DiT renderer, Vocos vocoder (134 MB, fp32)
model.int8.safetensors	The same model in int8, one fp16 scale per output channel (34 MB)
Silencio_demo.ipynb	Colab notebook with a Gradio demo: text to speech, voice cloning, voice conversion
config.json	Phoneme vocabulary, module sizes, chunking and the default voice
voices.safetensors	8 ready-made voice embeddings (256-d each)
ref_*.wav	The LibriSpeech clips those voices were enrolled from
silencio/	The inference code. Upstream modules are vendored under their Apache-2.0 licenses, with only import paths changed
requirements.txt	Runtime dependencies
samples/	The audio above
eval.json	The numbers in Results

Limitations

• English only, with American pronunciation. Numbers, abbreviations and rare words are pronounced only as well as misaki handles them.
• Output is 16 kHz, which is the rate of MeanVC2's vocoder. That is enough for voice assistants and audiobooks, but not for full-band audio.
• Prosody comes from Paradee's single training voice, so every voice speaks with similar timing and intonation. Speaker similarity is lower than in voice conversion, where timing comes from the source speaker.
• Long text is split into sentences that are spoken one after another, so prosody does not carry across sentences.

Responsible use

Do not use Silencio to impersonate a real person, to deceive listeners about who is speaking, or to clone or convert a voice without the speaker's consent. Tell listeners when speech is synthetic where that matters to them.

License

The weights and code in this repository are released under Apache 2.0, the license of every model they are built from:

• Paradee-8M v1.1 by Sahil Mahendrakar (Apache 2.0): text encoder.
• Kokoro-82M by hexgrad (Apache 2.0): the architecture and teacher behind Paradee, and the misaki G2P.
• MeanVC2 by ASLP-lab (Apache 2.0): DiT renderer, Vocos vocoder, Fast-U2++ encoder and the speaker-encoder code.
• Vocos by Gemelo AI (MIT): vocoder architecture.

The voice encoder used by enroll (WavLM-Large from UniSpeech, with its ECAPA-TDNN speaker-verification fine-tune) is not redistributed here. It is downloaded at runtime under its own license. The bundled voices come from LibriSpeech (CC BY 4.0) readers, whose recordings are from the public-domain LibriVox project.

Citation

@misc{silencio2026,
  title  = {Silencio: Zero-Shot Text-to-Speech by Connecting a Distilled Text Encoder to a Streaming Voice Renderer},
  author = {{BlueRose Labs}},
  year   = {2026},
  url    = {https://huggingface.co/BlueRoseLabs/Silencio}
}

@misc{mahendrakar2026paradee,
  title  = {Paradee: Distilling Kokoro-82M into an 8M-Parameter Single-Voice Text-to-Speech Model},
  author = {Mahendrakar, Sahil},
  year   = {2026},
  url    = {https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.0}
}

MeanVC2: github.com/ASLP-lab/MeanVC2.