--- license: apache-2.0 language: - en pipeline_tag: text-to-speech library_name: pytorch base_model: - sahilmahendrakar/Paradee-8M-v1.1 - ASLP-lab/MeanVC2 tags: - text-to-speech - tts - zero-shot - voice-cloning - voice-conversion - streaming - flow-matching - on-device --- # Silencio **Small zero-shot text-to-speech that speaks in any voice.** Silencio turns English text into 16 kHz speech in a voice it has never heard before. A few seconds of reference audio are enough. All 33.6M parameters fit in one 134 MB `model.safetensors`. It renders audio in 120 ms chunks and runs about 9x faster than real time on one CPU thread of an Apple M4. The same weights also do voice conversion: give it speech instead of text and it says the same words in another voice. [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1dp4ivPg2DGctGV7joEYKY_El8aIoBxuS) Silencio is built on two open models. The text front-end comes from [Paradee-8M v1.1](https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.1) by Sahil Mahendrakar, itself distilled from [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) by hexgrad. The voice renderer and vocoder come from [MeanVC2](https://huggingface.co/ASLP-lab/MeanVC2) by ASLP-lab. A new 3.4M-parameter content adapter connects them, so text goes straight into the renderer with no intermediate audio. ## Model details - **Model type:** zero-shot text-to-speech. A text encoder predicts phoneme durations, pitch and phonetic features. A ConvNeXt adapter turns them into content tokens at 25 Hz. A mean-flow DiT renders a mel spectrogram from those tokens and a 256-d voice embedding in 3 steps. Vocos then makes the waveform. - **Parameters:** 33.6M (text encoder 4.2M, adapter 3.4M, renderer 17.7M, vocoder 8.3M) - **Weights files:** `model.safetensors`, 134 MB (fp32), and `model.int8.safetensors`, 34 MB (int8, practically the same quality) - **Language:** English (American pronunciation) - **Voices:** zero-shot, from a 5–15 s reference clip. 8 ready-made voices are included. - **Output:** mono, 16 kHz - **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) ### How it fits together ``` text ─► misaki G2P ─► text encoder ─► content adapter ─► DiT renderer ─► Vocos ─► 16 kHz audio (phonemes) durations, F0, content tokens mean flow, ▲ features @40 Hz @25 Hz 3 steps │ ▲ │ voice embedding ──┘ (256-d) │ speech ─► Fast-U2++ streaming ASR encoder ─► content tokens ─┘ (voice conversion) ``` The text encoder predicts how long each phoneme lasts and how the pitch moves. The renderer only decides how it sounds in the chosen voice. So timing and intonation come from the text, and timbre comes from the voice embedding. ## Quickstart ### Try it in Colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1dp4ivPg2DGctGV7joEYKY_El8aIoBxuS) Open the [demo notebook](https://colab.research.google.com/drive/1dp4ivPg2DGctGV7joEYKY_El8aIoBxuS) and run all cells. It opens a Gradio demo with text to speech, voice cloning from the microphone or a file, and voice conversion. ### Install ```sh hf download BlueRoseLabs/Silencio --local-dir Silencio cd Silencio pip install -r requirements.txt pip install --no-deps --ignore-requires-python misaki==0.9.4 # G2P; declares python<3.13 but runs on 3.13 ``` ### Python ```python from silencio import SilencioTTS tts = SilencioTTS.from_pretrained("BlueRoseLabs/Silencio") # or the local folder tts.save("speech.wav", "Hello from Silencio.", voice="librispeech_1320") # Streaming: one chunk per sentence, float32 at 16 kHz for pcm in tts.stream("Speech can start playing right away. The next sentence follows.", voice="librispeech_2094"): ... ``` For the 34 MB int8 weights, pass `quant="int8"` to `from_pretrained`. They are dequantized on load, so speed and memory use are the same as fp32. `generate` returns a numpy array, and `speed` (default 1.0) changes the speaking rate. On first use misaki downloads spaCy's small English model (about 13 MB). ### Any voice from a short clip ```python import soundfile as sf from silencio.enroll import enroll # pip install s3prl voice = enroll(*sf.read("reference.wav")) # 256-d embedding; save it with torch.save and reuse it tts.save("cloned.wav", "This is my voice, read by Silencio.", voice=voice) ``` The voice encoder is WavLM-Large with ECAPA-TDNN, the same one MeanVC2 was trained with. It is large (about 1.3 GB) and only needed to add new voices, so it is not part of `model.safetensors`. It is downloaded on first use. A saved embedding needs nothing extra. ### Voice conversion ```python wav, sr = sf.read("source.wav") sf.write("converted.wav", tts.convert(wav, sr, voice="librispeech_2094"), 16000) ``` For speech input, the content tokens come from MeanVC2's streaming Fast-U2++ encoder. It is downloaded from [ASLP-lab/MeanVC2](https://huggingface.co/ASLP-lab/MeanVC2) on first use and runs on CPU. ## Voices `voices.safetensors` holds 8 voices enrolled from LibriSpeech test-clean readers. A reader's voice is called `librispeech_`. The reference clip used for each one is included as `ref_.wav`. | Voice | Reference clip | |---|---| | `librispeech_1320` (default) | [ref_1320.wav](ref_1320.wav) | | `librispeech_2094` | [ref_2094.wav](ref_2094.wav) | | `librispeech_260` | [ref_260.wav](ref_260.wav) | | `librispeech_3575` | [ref_3575.wav](ref_3575.wav) | | `librispeech_5639` | [ref_5639.wav](ref_5639.wav) | | `librispeech_6930` | [ref_6930.wav](ref_6930.wav) | | `librispeech_7127` | [ref_7127.wav](ref_7127.wav) | | `librispeech_7729` | [ref_7729.wav](ref_7729.wav) | ## Samples Five held-out sentences, read by Silencio in three voices. The last column is Paradee-8M v1.1, whose text encoder Silencio uses, in its single voice, for comparison. The sentences are in [`samples/sentences.txt`](samples/sentences.txt). | # | `librispeech_1320` | `librispeech_2094` | `librispeech_260` | Paradee-8M v1.1 | |---|---|---|---|---| | 1 | [listen](samples/1_librispeech_1320.wav) | [listen](samples/1_librispeech_2094.wav) | [listen](samples/1_librispeech_260.wav) | [listen](samples/1_paradee.wav) | | 2 | [listen](samples/2_librispeech_1320.wav) | [listen](samples/2_librispeech_2094.wav) | [listen](samples/2_librispeech_260.wav) | [listen](samples/2_paradee.wav) | | 3 | [listen](samples/3_librispeech_1320.wav) | [listen](samples/3_librispeech_2094.wav) | [listen](samples/3_librispeech_260.wav) | [listen](samples/3_paradee.wav) | | 4 | [listen](samples/4_librispeech_1320.wav) | [listen](samples/4_librispeech_2094.wav) | [listen](samples/4_librispeech_260.wav) | [listen](samples/4_paradee.wav) | | 5 | [listen](samples/5_librispeech_1320.wav) | [listen](samples/5_librispeech_2094.wav) | [listen](samples/5_librispeech_260.wav) | [listen](samples/5_paradee.wav) | Voice conversion: [source](ref_1320.wav) (LibriSpeech 1320) → [converted to `librispeech_2094`](samples/vc_1320_to_2094.wav). Whisper transcribes both identically, and the speaker similarity to the target voice is 0.83. ## Results All numbers are on Paradee's 200 held-out WikiText sentences, scored the way the Paradee card scores them. Silencio speaks them in its 8 voices in turn. | Model (voice) | Params | Weights | UTMOS | WER | Speaker similarity | Speed, T4 | Speed, 1 CPU thread | |---|---:|---:|---:|---:|---:|---:|---:| | **Silencio (8 zero-shot voices)** | **33.6M** | **134 MB** | 4.15 | 7.0% | 0.72 | 3.7x | 8.8x (M4) · 1.6x (Colab) | | **Silencio int8** | **33.6M** | **34 MB** | 4.12 | 7.2% | 0.72 | 4.0x | 1.5x (Colab) | | Paradee-8M v1.1 (af_heart, single voice), PyTorch | 7.9M | 8.5 MB (int8) | 4.41 | 6.3% | – | – | – | - **UTMOS** is a neural network trained on human ratings. It predicts how natural a clip sounds, on a scale from 1 to 5 (SpeechMOS `utmos22_strong`). - **WER** is the share of words that Whisper (base) transcribes wrongly. - **Speaker similarity** is the cosine similarity between the voice of the output and the voice of the reference clip, measured with MeanVC2's WavLM/ECAPA encoder. 1.0 is the same embedding. - **Speed** is how many times faster than real time the model runs, counting text to waveform, in PyTorch fp32. CPU speed is measured with one thread, on an Apple M4 (one two-sentence paragraph) and on the Colab host (10 held-out sentences). Paradee was scored by the same script, in PyTorch, which is why its WER differs a little from the 5.9% on its own card. Paradee speaks one studio voice at 24 kHz. Silencio speaks voices taken from audiobook recordings, and it does so at 16 kHz. Its UTMOS therefore also reflects the recording quality of the reference voices, not only the model. ## How it was made Only the adapter was trained. Everything else is reused frozen from the two base models. 1. **Content space.** MeanVC2's renderer takes content tokens from a streaming ASR encoder (256-d at 25 Hz) and a speaker embedding. Content tokens carry what is said and how, but not who says it. 2. **Targets.** Paradee read 6,000 WikiText sentences at speeds 0.9, 1.0 and 1.1 (about 6 hours). For each one, Paradee's internal text-side features were kept, and MeanVC2's ASR encoder was run on Paradee's audio to get the content tokens that audio would produce. 3. **Adapter.** A ConvNeXt network (4 blocks at 40 Hz, resampling to 25 Hz, then 4 more blocks) learns to map the text features, pitch and loudness to those content tokens. It uses an L1 plus cosine loss. Training took 15,000 steps at batch 32, about 22 minutes on one T4. 4. **One model.** The text encoder, adapter, renderer (EMA weights) and vocoder were put into a single `model.safetensors`. Weight norm was folded into the weights, and the Vocos TorchScript was converted to plain PyTorch. The output of the merged model matches the original chained pipeline to within 7e-4. In speech-to-speech mode the original ASR encoder takes the adapter's place, so voice conversion works exactly as in MeanVC2. ## Files | File | What it is | |---|---| | `model.safetensors` | The whole model: text encoder, content adapter, DiT renderer, Vocos vocoder (134 MB, fp32) | | `model.int8.safetensors` | The same model in int8, one fp16 scale per output channel (34 MB) | | `Silencio_demo.ipynb` | Colab notebook with a Gradio demo: text to speech, voice cloning, voice conversion | | `config.json` | Phoneme vocabulary, module sizes, chunking and the default voice | | `voices.safetensors` | 8 ready-made voice embeddings (256-d each) | | `ref_*.wav` | The LibriSpeech clips those voices were enrolled from | | `silencio/` | The inference code. Upstream modules are vendored under their Apache-2.0 licenses, with only import paths changed | | `requirements.txt` | Runtime dependencies | | `samples/` | The audio above | | `eval.json` | The numbers in Results | ## Limitations - English only, with American pronunciation. Numbers, abbreviations and rare words are pronounced only as well as misaki handles them. - Output is 16 kHz, which is the rate of MeanVC2's vocoder. That is enough for voice assistants and audiobooks, but not for full-band audio. - Prosody comes from Paradee's single training voice, so every voice speaks with similar timing and intonation. Speaker similarity is lower than in voice conversion, where timing comes from the source speaker. - Long text is split into sentences that are spoken one after another, so prosody does not carry across sentences. ## Responsible use Do not use Silencio to impersonate a real person, to deceive listeners about who is speaking, or to clone or convert a voice without the speaker's consent. Tell listeners when speech is synthetic where that matters to them. ## License The weights and code in this repository are released under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0), the license of every model they are built from: - [Paradee-8M v1.1](https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.1) by Sahil Mahendrakar (Apache 2.0): text encoder. - [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) by hexgrad (Apache 2.0): the architecture and teacher behind Paradee, and the [misaki](https://github.com/hexgrad/misaki) G2P. - [MeanVC2](https://huggingface.co/ASLP-lab/MeanVC2) by ASLP-lab (Apache 2.0): DiT renderer, Vocos vocoder, Fast-U2++ encoder and the speaker-encoder code. - [Vocos](https://github.com/gemelo-ai/vocos) by Gemelo AI (MIT): vocoder architecture. The voice encoder used by `enroll` (WavLM-Large from [UniSpeech](https://github.com/microsoft/UniSpeech), with its ECAPA-TDNN speaker-verification fine-tune) is not redistributed here. It is downloaded at runtime under its own license. The bundled voices come from [LibriSpeech](https://www.openslr.org/12) (CC BY 4.0) readers, whose recordings are from the public-domain LibriVox project. ## Citation ```bibtex @misc{silencio2026, title = {Silencio: Zero-Shot Text-to-Speech by Connecting a Distilled Text Encoder to a Streaming Voice Renderer}, author = {{BlueRose Labs}}, year = {2026}, url = {https://huggingface.co/BlueRoseLabs/Silencio} } @misc{mahendrakar2026paradee, title = {Paradee: Distilling Kokoro-82M into an 8M-Parameter Single-Voice Text-to-Speech Model}, author = {Mahendrakar, Sahil}, year = {2026}, url = {https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.0} } ``` MeanVC2: [github.com/ASLP-lab/MeanVC2](https://github.com/ASLP-lab/MeanVC2).