| --- |
| license: apache-2.0 |
| language: |
| - en |
| pipeline_tag: text-to-speech |
| library_name: pytorch |
| base_model: |
| - sahilmahendrakar/Paradee-8M-v1.1 |
| - ASLP-lab/MeanVC2 |
| tags: |
| - text-to-speech |
| - tts |
| - zero-shot |
| - voice-cloning |
| - voice-conversion |
| - streaming |
| - flow-matching |
| - on-device |
| --- |
| |
| # Silencio |
|
|
| **Small zero-shot text-to-speech that speaks in any voice.** |
|
|
| Silencio turns English text into 16 kHz speech in a voice it has never heard before. A few seconds |
| of reference audio are enough. All 33.6M parameters fit in one 134 MB `model.safetensors`. It renders |
| audio in 120 ms chunks and runs about 9x faster than real time on one CPU thread of an Apple M4. |
|
|
| The same weights also do voice conversion: give it speech instead of text and it says the same words in |
| another voice. |
|
|
| [](https://colab.research.google.com/drive/1dp4ivPg2DGctGV7joEYKY_El8aIoBxuS) |
|
|
| Silencio is built on two open models. The text front-end comes from |
| [Paradee-8M v1.1](https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.1) by Sahil Mahendrakar, |
| itself distilled from [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) by hexgrad. The |
| voice renderer and vocoder come from [MeanVC2](https://huggingface.co/ASLP-lab/MeanVC2) by ASLP-lab. |
| A new 3.4M-parameter content adapter connects them, so text goes straight into the renderer with no |
| intermediate audio. |
|
|
| ## Model details |
|
|
| - **Model type:** zero-shot text-to-speech. A text encoder predicts phoneme durations, pitch and |
| phonetic features. A ConvNeXt adapter turns them into content tokens at 25 Hz. A mean-flow DiT |
| renders a mel spectrogram from those tokens and a 256-d voice embedding in 3 steps. Vocos then |
| makes the waveform. |
| - **Parameters:** 33.6M (text encoder 4.2M, adapter 3.4M, renderer 17.7M, vocoder 8.3M) |
| - **Weights files:** `model.safetensors`, 134 MB (fp32), and `model.int8.safetensors`, 34 MB (int8, practically the same quality) |
| - **Language:** English (American pronunciation) |
| - **Voices:** zero-shot, from a 5–15 s reference clip. 8 ready-made voices are included. |
| - **Output:** mono, 16 kHz |
| - **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
|
|
| ### How it fits together |
|
|
| ``` |
| text ─► misaki G2P ─► text encoder ─► content adapter ─► DiT renderer ─► Vocos ─► 16 kHz audio |
| (phonemes) durations, F0, content tokens mean flow, ▲ |
| features @40 Hz @25 Hz 3 steps │ |
| ▲ │ |
| voice embedding ──┘ (256-d) │ |
| speech ─► Fast-U2++ streaming ASR encoder ─► content tokens ─┘ (voice conversion) |
| ``` |
|
|
| The text encoder predicts how long each phoneme lasts and how the pitch moves. The renderer only |
| decides how it sounds in the chosen voice. So timing and intonation come from the text, and timbre |
| comes from the voice embedding. |
|
|
| ## Quickstart |
|
|
| ### Try it in Colab |
|
|
| [](https://colab.research.google.com/drive/1dp4ivPg2DGctGV7joEYKY_El8aIoBxuS) |
|
|
| Open the [demo notebook](https://colab.research.google.com/drive/1dp4ivPg2DGctGV7joEYKY_El8aIoBxuS) and run all cells. It opens a Gradio demo with text to speech, voice cloning |
| from the microphone or a file, and voice conversion. |
|
|
| ### Install |
|
|
| ```sh |
| hf download BlueRoseLabs/Silencio --local-dir Silencio |
| cd Silencio |
| pip install -r requirements.txt |
| pip install --no-deps --ignore-requires-python misaki==0.9.4 # G2P; declares python<3.13 but runs on 3.13 |
| ``` |
|
|
| ### Python |
|
|
| ```python |
| from silencio import SilencioTTS |
| |
| tts = SilencioTTS.from_pretrained("BlueRoseLabs/Silencio") # or the local folder |
| tts.save("speech.wav", "Hello from Silencio.", voice="librispeech_1320") |
| |
| # Streaming: one chunk per sentence, float32 at 16 kHz |
| for pcm in tts.stream("Speech can start playing right away. The next sentence follows.", voice="librispeech_2094"): |
| ... |
| ``` |
|
|
| For the 34 MB int8 weights, pass `quant="int8"` to `from_pretrained`. They are dequantized on load, so speed and |
| memory use are the same as fp32. |
|
|
| `generate` returns a numpy array, and `speed` (default 1.0) changes the speaking rate. On first use misaki downloads spaCy's small English model (about 13 MB). |
|
|
| ### Any voice from a short clip |
|
|
| ```python |
| import soundfile as sf |
| from silencio.enroll import enroll # pip install s3prl |
| |
| voice = enroll(*sf.read("reference.wav")) # 256-d embedding; save it with torch.save and reuse it |
| tts.save("cloned.wav", "This is my voice, read by Silencio.", voice=voice) |
| ``` |
|
|
| The voice encoder is WavLM-Large with ECAPA-TDNN, the same one MeanVC2 was trained with. It is large |
| (about 1.3 GB) and only needed to add new voices, so it is not part of `model.safetensors`. It is |
| downloaded on first use. A saved embedding needs nothing extra. |
|
|
| ### Voice conversion |
|
|
| ```python |
| wav, sr = sf.read("source.wav") |
| sf.write("converted.wav", tts.convert(wav, sr, voice="librispeech_2094"), 16000) |
| ``` |
|
|
| For speech input, the content tokens come from MeanVC2's streaming Fast-U2++ encoder. It is |
| downloaded from [ASLP-lab/MeanVC2](https://huggingface.co/ASLP-lab/MeanVC2) on first use and runs on CPU. |
|
|
| ## Voices |
|
|
| `voices.safetensors` holds 8 voices enrolled from LibriSpeech test-clean readers. A reader's voice |
| is called `librispeech_<speaker id>`. The reference clip used for each one is included as `ref_<id>.wav`. |
|
|
| | Voice | Reference clip | |
| |---|---| |
| | `librispeech_1320` (default) | [ref_1320.wav](ref_1320.wav) | |
| | `librispeech_2094` | [ref_2094.wav](ref_2094.wav) | |
| | `librispeech_260` | [ref_260.wav](ref_260.wav) | |
| | `librispeech_3575` | [ref_3575.wav](ref_3575.wav) | |
| | `librispeech_5639` | [ref_5639.wav](ref_5639.wav) | |
| | `librispeech_6930` | [ref_6930.wav](ref_6930.wav) | |
| | `librispeech_7127` | [ref_7127.wav](ref_7127.wav) | |
| | `librispeech_7729` | [ref_7729.wav](ref_7729.wav) | |
|
|
| ## Samples |
|
|
| Five held-out sentences, read by Silencio in three voices. The last column is |
| Paradee-8M v1.1, whose text encoder Silencio uses, in its single voice, for comparison. The sentences are |
| in [`samples/sentences.txt`](samples/sentences.txt). |
|
|
| | # | `librispeech_1320` | `librispeech_2094` | `librispeech_260` | Paradee-8M v1.1 | |
| |---|---|---|---|---| |
| | 1 | [listen](samples/1_librispeech_1320.wav) | [listen](samples/1_librispeech_2094.wav) | [listen](samples/1_librispeech_260.wav) | [listen](samples/1_paradee.wav) | |
| | 2 | [listen](samples/2_librispeech_1320.wav) | [listen](samples/2_librispeech_2094.wav) | [listen](samples/2_librispeech_260.wav) | [listen](samples/2_paradee.wav) | |
| | 3 | [listen](samples/3_librispeech_1320.wav) | [listen](samples/3_librispeech_2094.wav) | [listen](samples/3_librispeech_260.wav) | [listen](samples/3_paradee.wav) | |
| | 4 | [listen](samples/4_librispeech_1320.wav) | [listen](samples/4_librispeech_2094.wav) | [listen](samples/4_librispeech_260.wav) | [listen](samples/4_paradee.wav) | |
| | 5 | [listen](samples/5_librispeech_1320.wav) | [listen](samples/5_librispeech_2094.wav) | [listen](samples/5_librispeech_260.wav) | [listen](samples/5_paradee.wav) | |
|
|
| Voice conversion: [source](ref_1320.wav) (LibriSpeech 1320) → [converted to `librispeech_2094`](samples/vc_1320_to_2094.wav). Whisper transcribes both identically, and the speaker similarity to the target voice is 0.83. |
|
|
| ## Results |
|
|
| All numbers are on Paradee's 200 held-out WikiText sentences, scored the way the Paradee card scores |
| them. Silencio speaks them in its 8 voices in turn. |
|
|
| | Model (voice) | Params | Weights | UTMOS | WER | Speaker similarity | Speed, T4 | Speed, 1 CPU thread | |
| |---|---:|---:|---:|---:|---:|---:|---:| |
| | **Silencio (8 zero-shot voices)** | **33.6M** | **134 MB** | 4.15 | 7.0% | 0.72 | 3.7x | 8.8x (M4) · 1.6x (Colab) | |
| | **Silencio int8** | **33.6M** | **34 MB** | 4.12 | 7.2% | 0.72 | 4.0x | 1.5x (Colab) | |
| | Paradee-8M v1.1 (af_heart, single voice), PyTorch | 7.9M | 8.5 MB (int8) | 4.41 | 6.3% | – | – | – | |
| |
| - **UTMOS** is a neural network trained on human ratings. It predicts how natural a clip sounds, |
| on a scale from 1 to 5 (SpeechMOS `utmos22_strong`). |
| - **WER** is the share of words that Whisper (base) transcribes wrongly. |
| - **Speaker similarity** is the cosine similarity between the voice of the output and the voice of the |
| reference clip, measured with MeanVC2's WavLM/ECAPA encoder. 1.0 is the same embedding. |
| - **Speed** is how many times faster than real time the model runs, counting text to waveform, in PyTorch |
| fp32. CPU speed is measured with one thread, on an Apple M4 (one two-sentence paragraph) and on the Colab host (10 held-out sentences). |
|
|
| Paradee was scored by the same script, in PyTorch, which is why its WER differs a little from the 5.9% on its own card. |
|
|
| Paradee speaks one studio voice at 24 kHz. Silencio speaks voices taken from audiobook recordings, and |
| it does so at 16 kHz. Its UTMOS therefore also reflects the recording quality of the reference voices, |
| not only the model. |
|
|
| ## How it was made |
|
|
| Only the adapter was trained. Everything else is reused frozen from the two base models. |
|
|
| 1. **Content space.** MeanVC2's renderer takes content tokens from a streaming ASR encoder (256-d at |
| 25 Hz) and a speaker embedding. Content tokens carry what is said and how, but not who says it. |
| 2. **Targets.** Paradee read 6,000 WikiText sentences at speeds 0.9, 1.0 and 1.1 (about 6 hours). For |
| each one, Paradee's internal text-side features were kept, and MeanVC2's ASR encoder was run on |
| Paradee's audio to get the content tokens that audio would produce. |
| 3. **Adapter.** A ConvNeXt network (4 blocks at 40 Hz, resampling to 25 Hz, then 4 more blocks) learns to |
| map the text features, pitch and loudness to those content tokens. It uses an L1 plus cosine loss. |
| Training took 15,000 steps at batch 32, about 22 minutes on one T4. |
| 4. **One model.** The text encoder, adapter, renderer (EMA weights) and vocoder were put into a single |
| `model.safetensors`. Weight norm was folded into the weights, and the Vocos TorchScript was converted |
| to plain PyTorch. The output of the merged model matches the original chained pipeline to within |
| 7e-4. |
|
|
| In speech-to-speech mode the original ASR encoder takes the adapter's place, so voice conversion works |
| exactly as in MeanVC2. |
|
|
| ## Files |
|
|
| | File | What it is | |
| |---|---| |
| | `model.safetensors` | The whole model: text encoder, content adapter, DiT renderer, Vocos vocoder (134 MB, fp32) | |
| | `model.int8.safetensors` | The same model in int8, one fp16 scale per output channel (34 MB) | |
| | `Silencio_demo.ipynb` | Colab notebook with a Gradio demo: text to speech, voice cloning, voice conversion | |
| | `config.json` | Phoneme vocabulary, module sizes, chunking and the default voice | |
| | `voices.safetensors` | 8 ready-made voice embeddings (256-d each) | |
| | `ref_*.wav` | The LibriSpeech clips those voices were enrolled from | |
| | `silencio/` | The inference code. Upstream modules are vendored under their Apache-2.0 licenses, with only import paths changed | |
| | `requirements.txt` | Runtime dependencies | |
| | `samples/` | The audio above | |
| | `eval.json` | The numbers in Results | |
|
|
| ## Limitations |
|
|
| - English only, with American pronunciation. Numbers, abbreviations and rare words are pronounced only |
| as well as misaki handles them. |
| - Output is 16 kHz, which is the rate of MeanVC2's vocoder. That is enough for voice assistants |
| and audiobooks, but not for full-band audio. |
| - Prosody comes from Paradee's single training voice, so every voice speaks with similar timing and |
| intonation. Speaker similarity is lower than in voice conversion, where timing comes from the |
| source speaker. |
| - Long text is split into sentences that are spoken one after another, so prosody does not carry |
| across sentences. |
|
|
| ## Responsible use |
|
|
| Do not use Silencio to impersonate a real person, to deceive listeners about who is speaking, or |
| to clone or convert a voice without the speaker's consent. Tell listeners when speech is synthetic where |
| that matters to them. |
|
|
| ## License |
|
|
| The weights and code in this repository are released under |
| [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0), the license of every model they are built |
| from: |
|
|
| - [Paradee-8M v1.1](https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.1) by Sahil Mahendrakar |
| (Apache 2.0): text encoder. |
| - [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) by hexgrad (Apache 2.0): the architecture and |
| teacher behind Paradee, and the [misaki](https://github.com/hexgrad/misaki) G2P. |
| - [MeanVC2](https://huggingface.co/ASLP-lab/MeanVC2) by ASLP-lab (Apache 2.0): DiT renderer, Vocos |
| vocoder, Fast-U2++ encoder and the speaker-encoder code. |
| - [Vocos](https://github.com/gemelo-ai/vocos) by Gemelo AI (MIT): vocoder architecture. |
|
|
| The voice encoder used by `enroll` (WavLM-Large from |
| [UniSpeech](https://github.com/microsoft/UniSpeech), with its ECAPA-TDNN speaker-verification |
| fine-tune) is not redistributed here. It is downloaded at runtime under its own license. The bundled |
| voices come from [LibriSpeech](https://www.openslr.org/12) (CC BY 4.0) readers, whose recordings are from |
| the public-domain LibriVox project. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{silencio2026, |
| title = {Silencio: Zero-Shot Text-to-Speech by Connecting a Distilled Text Encoder to a Streaming Voice Renderer}, |
| author = {{BlueRose Labs}}, |
| year = {2026}, |
| url = {https://huggingface.co/BlueRoseLabs/Silencio} |
| } |
| |
| @misc{mahendrakar2026paradee, |
| title = {Paradee: Distilling Kokoro-82M into an 8M-Parameter Single-Voice Text-to-Speech Model}, |
| author = {Mahendrakar, Sahil}, |
| year = {2026}, |
| url = {https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.0} |
| } |
| ``` |
|
|
| MeanVC2: [github.com/ASLP-lab/MeanVC2](https://github.com/ASLP-lab/MeanVC2). |
|
|
