mmmhusto TTS English
A 56 M-parameter English text-to-speech model that reads and talks. Written and trained from scratch, small enough to read in an afternoon, fast enough for a laptop GPU, and open about everything: code, data, training runs and the experiments that failed.
- Two speaking styles in every voice: audiobook reading and everyday conversation, with a strength knob.
- Three studio voices (two female, one male, 26 to 30 hours each) plus 589 more.
- Small: 42 M acoustic model + 14 M vocoder, the vocoder fine-tuned on this model's own output. The whole model is one file of plain PyTorch (~414 lines).
- Fast mode: a distilled decoder needs 2 passes instead of 20 and sounded the same in a blind test. With the
built-in Vocos vocoder it runs 6x faster than real time on a 2017 dual-core i3 and 9 to 18x on an M1 Pro CPU on a laptop CPU and needs nothing but
pip install. - Reads what people write: numbers, money, times, dates, units and abbreviations ("Dr. Smith, $45.50, 9:30 a.m., 2.5 km") are written out before the phonemes.
- Voice cloning (experimental): a voice from 5 to 15 s of someone's speech, in a tenth of a second.
- Any length:
tts.long(text)andtts.stream(text)read a whole article sentence by sentence. - Controllable: pace, comma pauses, the gap between sentences, style strength, seed.
- Cheap to adapt: the conversational style was added in 30 minutes on one RTX 3080 (12 GB).
- Try it in your browser, nothing to install: the demo runs the model on your own device (WebGPU or WebAssembly; 210 MB on the first visit, then cached).
- A local web page to try it:
python app.py.
Listen
| Voice 92 (female, default) | Voice 11697 (male) | Voice 8051 (female) | |
|---|---|---|---|
| Reading a story |
The same text in both styles ("Are you kidding me? We missed the train again? Okay, okay, I get it."):
| Reading | Conversational | |
|---|---|---|
| Voice 92 | ||
| Voice 11697 |
Numbers and fast mode (voice 92):
| "Dr. Smith will see you at 9:30 on March 3rd; the visit costs $45.50, and the clinic is 2.5 km away." | |
The story above in fast mode (fast=True, 2 decoder passes instead of 20) |
All clips are exactly what the code below produces with BigVGAN (voice 11697 conversational at style_scale=1.0).
Status, honestly. On automatic measures (intelligibility, signal quality) the model scores like larger open models. By ear it is not there yet: the author's listening puts it somewhat below the best open English TTS, mostly in naturalness. The numbers below are a floor, not a verdict; listen to the samples and judge for yourself. What we are working on next is at the end of this page.
Quick start
pip install torch numpy soundfile huggingface_hub safetensors g2p_en
python -c "from g2p_en import G2p; G2p()" # fetches g2p_en's NLTK data once
# for the best quality also the BigVGAN vocoder's code (MIT):
git clone https://github.com/NVIDIA/BigVGAN && pip install librosa matplotlib # BigVGAN imports the last two
from huggingface_hub import snapshot_download
import sys, soundfile as sf
path = snapshot_download("mmmhusto/TTS-English")
sys.path.insert(0, path)
from tts_english import TTS, SAMPLE_RATE
tts = TTS.from_pretrained(path, bigvgan_dir="BigVGAN") # best quality
# tts = TTS.from_pretrained(path, fast=True) # no BigVGAN clone: Vocos + 2-pass decoder, fast on a CPU
wav = tts("It was not the answer she expected; still, she thanked him and left the room.")
sf.write("out.wav", wav, SAMPLE_RATE)
or python infer.py "Text to speak." out.wav (--fast, --vocoder vocos).
Try it in the browser. python app.py --bigvgan BigVGAN (or python app.py --vocoder vocos --fast) starts a local page at http://localhost:8790: type any
English text, pick a voice and set the gap between sentences, the comma pause, the pace, the timbre smoothing and the
seed. It uses an NVIDIA GPU or the Apple GPU (MPS) when there is one, else the CPU; on an Apple M-series GPU 8 s of
speech take about 5 s.
Which voice. Use the studio voices: en_hifi_92_clean (default, female), en_hifi_8051_other (female) and
en_hifi_11697_other (male). The model heard each of them for 26 to 30 hours. tts.voices lists all 592; 585 of the
others are LibriTTS-R readers with 0.1 to 0.5 hours each: they work, and they sound duller and less steady (see
"One steady voice" below). Keyword arguments override the delivery in config.json
(guidance, length_scale, pause_scale, temperature, dur_temperature, n_timesteps); seed=None gives a
different rendition every call.
Styles. tts(text, style="conversational") speaks the text as a reply in a conversation instead of reading it
aloud; style_scale (default 1.5) sets how strongly. Every voice can use either style; without style a voice keeps
the style it was recorded in (reading for all but en_dt_0 and en_dt_1). See "Speaking styles" below.
Pauses. pause_scale (default 2.5) stretches the pauses at commas, semicolons and colons. The pause after a
sentence end (. ! ?) is not stretched: it is capped at sentence_gap seconds (default 0.35). The audiobooks taught
the model about a second of silence between sentences, which the stretch used to make even longer; in a 3-sentence
test the gaps went from 0.85 s to about 0.45 s, including the decay of the last word. sentence_gap=None restores
the earlier behaviour (sentence ends stretched by pause_scale like commas).
Fast mode. TTS.from_pretrained(..., fast=True) swaps in a distilled decoder (fast_decoder.safetensors) that
makes 2 passes instead of 20 (10 steps with classifier-free guidance); the acoustic model gets about 10x faster. In a
blind test of 18 pairs (same text, voice and noise) the listener preferred fast mode 5 times, the default 6 times and
heard no difference 7 times. Over 90 sentences in 3 voices PESQ is 3.83 against 3.89 and word errors 1.5 % against
1.6 %; the one measured loss is a slightly flatter melody in the conversational style (2.77 against 2.88 st on
average). See "Fewer steps" below.
Vocoders. The default vocoder is BigVGAN, fine-tuned on this model's mels. vocoder="vocos" uses a Vocos vocoder
built into the package instead: no BigVGAN clone, and 25 to 170x faster on a CPU, at a cost in sound. See "The
vocoder" below; on a GPU keep BigVGAN.
Hardware and speed
Measured on 5 sentences (21 s of speech), voice 92; "x real time" is seconds of speech per second of computing (above 1
is faster than real time). Fast mode is the distilled 2-pass decoder (fast=True).
| Hardware | Default | Fast mode | Fast mode + Vocos | Memory |
|---|---|---|---|---|
| NVIDIA RTX 3080 (12 GB) | 9.9x real time | 29.5x | 57x | 0.36 GB VRAM (0.23 with Vocos), 1.7 GB RAM |
| Apple M1 Pro GPU (MPS) | 2.2x | 3.7x | 36x | 1.3 GB GPU memory, 0.7 GB RAM |
| Apple M1 Pro CPU, 8 threads | 0.5x | 0.6x | 9.4x (18x on 1 thread) | 1.3 GB RAM |
| Intel Core i3-7100 CPU (2 cores, 2017) | 0.3x | 0.6x | 6.0x | 1.3 GB RAM |
On a GPU use BigVGAN (the first two columns): it sounds cleaner and is fast enough. On a CPU BigVGAN is the slow part
(about 45 of 50 s on the M1 Pro), so there only Vocos runs faster than real time. Vocos is also faster with fewer
threads (torch.set_num_threads(1)).
Minimum: any machine that runs PyTorch, with 2 GB of free RAM; on a GPU 1 GB of VRAM is enough. Disk: 168 MB model, 140 MB fast decoder, 56 MB BigVGAN, 54 MB Vocos and 17 MB voice encoder, plus PyTorch itself.
One steady voice
The first samples of this model used LibriTTS-R voices and sounded dull and unsteady. Measuring what a listener hears, over 30 sentences per voice, showed why, and none of the fixes needed training:
| Highs above 6 kHz | Colour drift between sentences | Pitch drift | Start of the clip | |
|---|---|---|---|---|
| LibriTTS-R voices (0.5 h each) | −22 dB | ±140 Hz | ±8 to 20 Hz | 80 ms |
| Studio voice 92, raw | −16 dB | ±183 Hz | ±9 Hz | 80 ms |
| Studio voice 92, default pipeline | −16 dB | ±113 Hz | ±9 Hz | 150 ms of silence |
- The voice. LibriTTS-R itself has about 10 dB less energy above 10 kHz than the HiFi-TTS studio readers, even as real recordings through the vocoder, and its pauses are about 25 dB noisier.
- The colour. The flow model draws a slightly different "recording" of the voice for every sentence.
match_timbre(intts_english/post.py) pulls each sentence's long-term spectrum towards the voice's average profile, stored inconfig.jsonfor the three studio voices. On held-out sentences the drift fell from ±183 to ±101 Hz with PESQ unchanged (3.95 to 3.98). Real recordings of the same reader drift too (±178 Hz within one book), because a sentence full of "s" is simply brighter: part of this "drift" is the text, not the model. - The clip.
finishcuts the speech to its edges, fades it, adds clean silence and brings every sentence to one loudness.tts(text, raw=True)returns the untouched vocoder output. - What did not help: lower temperature and stronger guidance left the drift unchanged; a fixed noise seed helped a
little (it is the default:
seed=0).
Speaking styles
The audiobook voices only ever heard reading, so the model could only read. A style vector, added to the voice vector like a second voice, now carries how a sentence is said; the reading vector is fixed at zero, so reading is exactly the model as it was. The conversational vector was learned from DailyTalk (2 actors, 21.5 h of acted everyday dialogue, 2 541 conversations) alongside the three studio voices, on one RTX 3080 in 0.5 h.
Measured over 60 everyday replies per voice, with the package's default pipeline (melody = spread of the pitch, range = 5th to 95th percentile, both in semitones; pitch = median, pYIN):
| Reading | Conversational ×1.0 | Conversational ×1.5 | |
|---|---|---|---|
| Voice 92 (female): melody / range | 2.60 / 7.8 st | 3.26 / 9.8 st | 3.33 / 10.3 st |
| pitch, PESQ | 215 Hz, 3.81 | 230 Hz, 3.86 | 223 Hz, 3.78 |
| Voice 11697 (male): melody / range | 2.95 / 9.4 st | 3.77 / 11.5 st | 4.13 / 12.5 st |
| pitch, PESQ | 173 Hz, 3.91 | 170 Hz, 3.77 | 156 Hz, 3.79 |
| Voice 8051 (female): melody / range | 2.12 / 6.5 st | 2.29 / 7.3 st | 2.38 / 7.8 st |
| pitch, PESQ | 234 Hz, 4.02 | 237 Hz, 4.02 | 235 Hz, 3.99 |
Word errors (Whisper small.en) stay between 0.6 and 1.8 % everywhere. Real DailyTalk recordings have a melody of 3.4 to 4.2 st and a range of 11 to 13 st. Voice 8051 barely changes: the style works much better on some voices than on others. Reading quality is unchanged on the 30-sentence bench (PESQ 3.81 before, 3.82 after; word errors 1.8 % both), with a slightly flatter melody (2.75 → 2.56 st).
What did not work. The first try kept the new voices' vectors almost fixed (a tenth of the learning rate). The style
then also learned who was speaking: the male studio voice fell from 170 to 126 Hz, the pitch of the DailyTalk man, and
PESQ dropped to 3.60. Letting the voice vectors learn at full speed kept each voice itself; the style came out weaker,
and at strength 1.0 the voices keep their pitch (the male voice within 3 Hz). At 1.5 the melody grows a little more but
the male voice speaks 1.5 semitones lower; at 2.0 it drops 3 semitones. Without post-processing the style also brightens the voices (15 to
30 % higher spectral centroid), the sound of the DailyTalk studio; with match_timbre (on by default) the change stays
under 8 %.
The vocoder
BigVGAN learned to turn mels of real recordings into sound; the mels this model draws are slightly different. We rendered 12.3 h of the training audio through the model itself (each clip's own text, voice and alignment, the noise partly kept from the real mel) and fine-tuned the 14 M BigVGAN base on those mels against the real audio, with 30 % real mels mixed in, for 22 500 steps (2.2 h on one RTX 3080).
| Over 90 sentences in 3 voices | Published BigVGAN | Fine-tuned BigVGAN (default) | Fine-tuned Vocos |
|---|---|---|---|
| PESQ (SQUIM) | 3.89 | 3.96 | 3.19 (BigVGAN 3.95 on the same clips) |
| Word errors (Whisper small.en) | same | same | same |
| Blind test, 18 pairs: preferred : other : no difference | 10 : 0 : 8 against the published one | 0 : 9 : 9 against fine-tuned BigVGAN | |
| CPU time for 22 s of speech (M1 Pro) | 33 to 46 s | 33 to 46 s | 0.3 to 1.3 s |
Vocos (Siuzdak 2023) predicts a short-time spectrum per frame and makes the sound with one inverse FFT instead of
upsampling sample by sample. We started from charactr/vocos-mel-24khz (13.5 M, MIT) and fine-tuned it on the same
pairs in our mel format (30 000 steps, 3.3 h on the RTX 3080). It keeps every word, the melody and the colour of the
voice, but it sounds less clean (PESQ 0.6 to 1.0 lower in every voice and style, and the listener never preferred it).
It is there for CPUs, where BigVGAN cannot keep up with real time; on a GPU use BigVGAN. Its code is in
tts_english/vocos.py (no vocos package needed).
Fewer steps
The default decoder makes 20 passes per sentence: 10 flow steps, each twice for classifier-free guidance. Reflow distillation (Liu et al., 2022) teaches a copy of the decoder the straight path from noise to the teacher's result: the teacher (this model, default settings) rendered 27 000 sentences (32.4 h of mel) from fixed noise, and the student decoder learned to reach the same mel from the same noise in a few steps without guidance (20 000 steps, 2.25 h on one RTX 3080; the text encoder and durations are untouched).
| Over 90 sentences in 3 voices | Passes | PESQ | Word errors | Melody |
|---|---|---|---|---|
| Default | 20 | 3.89 | 1.6 % | 2.88 st |
| Default settings with 4 steps, no guidance | 4 | 3.89 | 1.4 % | 2.86 st |
| Fast mode (distilled, 2 steps) | 2 | 3.83 | 1.5 % | 2.77 st |
| Distilled, 1 step | 1 | 3.74 | 1.5 % | 2.70 st |
In a blind test of 18 pairs fast mode won 5 times, the default 6 times, and 7 times there was no difference.
Voice cloning (experimental)
voice = tts.clone_voice(wav, sample_rate) # instant
voice = tts.clone_voice(wav, sample_rate, text="what it says") # closer; 40 to 60 s on a laptop CPU
tts("Any text.", voice=voice, style="conversational")
A 4.3 M voice encoder (voice_encoder.safetensors) turns the mel of a recording into a voice vector in the place of
the model's voice table; the acoustic model is unchanged. It was trained through the frozen model: the vector made
from one clip of a speaker must let the model reproduce another clip of that speaker (LibriTTS-R, about 460 speakers,
0.2 h on one RTX 3080). With text, the vector is then refined against the recording itself for 200 steps.
Speaker similarity (SpeechBrain ECAPA, cosine to the speaker's real clips; 20 speakers the model never heard and 20 it did, 3 sentences each):
| Unheard speakers | Heard speakers | |
|---|---|---|
| Another real clip of the same person (ceiling) | 0.68 | 0.65 |
| Real clips of other people (chance) | 0.18 | 0.18 |
| The model's average voice | 0.15 | 0.15 |
| Cloned, instant | 0.34 | 0.34 |
| Cloned and refined with the transcript | 0.43 | 0.43 |
| The speaker's own row of the voice table | – | 0.37 |
So: a voice in the direction of the speaker, not a copy. It varies a lot between people (0.05 to 0.6) and works best on clean read speech. The ceiling today is the frozen acoustic model, whose voices were learned from 6 to 30 minutes each. Only clone a voice you have the right to use, and never to deceive. The output is not watermarked.
Text normalization
g2p_en reads digits on its own, and badly: "45" becomes the single unknown word "forty-five" and comes out as
"fortivwid", "1,234" as "twelve thirty-four". tts_english/normalize.py writes out every number before the phonemes,
with plain rules: money ("$45.50" → "forty-five dollars and fifty cents", "$1.5 million"), times ("9:30 p.m."),
dates ("March 3rd", "12/03/1985"), years and decades ("1968", "the '90s"), ordinals, decimals, ranges, percent and
units ("2.5 km", "37°C"), abbreviations ("Dr.", "St.", "Apt."), Roman numerals after names and keywords ("Henry
VIII", "Chapter IV") and acronyms ("FBI" spelled, "NASA" said). Over 60 number-heavy sentences Whisper's word errors
fell from 8.6 % to 5.9 %; most of the rest are the scorer writing "Dr" or "12.30".
Architecture
All of it is in tts_english/model.py, ~414 lines of plain PyTorch, no vendored code.
- Text encoder. Six pre-norm transformer layers (256 wide) with rotary positions over ARPAbet phonemes from g2p_en. It also predicts a coarse mel per phoneme, which drives the alignment.
- Alignment. Monotonic alignment search during training, so no external aligner is needed.
- Durations (v2.1). A small flow-matching head samples the length of every phoneme and pause. It replaces the
log-domain regression, which predicts the geometric mean: on held-out clips that made speech 14 % shorter than the
data and sentence-final pauses 36 % shorter. The regression head stays as a deterministic fallback
(
dur_temperature=None). - Decoder. A diffusion transformer: 10 layers, 512 wide, adaptive layer norm on time and speaker. It is trained with conditional flow matching and reads the full encoder states, not the blurry per-phoneme mel.
- Decoder input (v2.1). Every frame also gets its position inside its phoneme and the phoneme's length.
- Guidance. The text is dropped for 10 % of the training clips, so the sampler can use classifier-free guidance (1.5 by default). This makes speech clearer and pronunciation more exact.
- Vocoder. BigVGAN base, 14 M, the same 100-band 24 kHz mel format as the 112 M BigVGAN-v2 we computed the features with, fine-tuned on this model's mels; or Vocos for CPUs (see "The vocoder").
- Fast decoder. A reflow-distilled copy of the decoder for 2 steps without guidance (see "Fewer steps").
- Weights. The exported weights are an exponential moving average of the training weights.
Data
- Sources:
- LibriTTS-R: audiobook speech, restored, 24 kHz.
- HiFi-TTS: studio audiobooks from 10 readers, resampled to 24 kHz.
- Both are CC BY 4.0.
- DailyTalk (copy on the Hub:
eustlb/dailytalk): acted everyday dialogue, 2 speakers, 21.5 h, 24 kHz, CC BY-SA 4.0. Used only for the conversational style (v2.2); its two speakers are the voicesen_dt_0(male) anden_dt_1(female).
- Features: mels, phonemes and a quality score for every 12th clip. They are public as mmmhusto/sktts-en-features: 455 183 clips, 525 hours.
- Training subset: the best-sounding 60 % of the speakers by the SQUIM SI-SDR estimate, which leaves 590 voices and 219 832 clips.
- Loudness: every clip is levelled to the same loudness before the mel.
- No other data: the model saw no other data, and no data in any other language.
Training
| Run | Steps | Start | What changed | Hardware, time |
|---|---|---|---|---|
| v2-en | 25 000 | from scratch | the v2 architecture | 1x AMD MI300X |
| v2-en-long | +50 000 | v2-en | cosine schedule, EMA 0.9999, 160 k frames per batch | MI300X, 6.3 h |
| v2.1 | +20 000 | v2-en-long, step 50 000 | phone position, sampled durations, 20 % uniform flow times | MI300X, 2.7 h |
| v2.2 (this model) | +6 000 | v2.1 | speaking styles: studio voices 92, 8051, 11697 + DailyTalk (twice), voice vectors at full learning rate, lr 1e-4 | 1x RTX 3080 12 GB, 0.5 h |
| v2.3: vocoder | 22 500 | BigVGAN base | fine-tuned on the model's mels (12.3 h) | RTX 3080, 2.2 h |
| v2.3: Vocos | 30 000 | charactr/vocos-mel-24khz | the same pairs, our mel format | RTX 3080, 3.3 h |
| v2.3: fast decoder | 20 000 | v2.2 decoder | reflow on 32.4 h of the model's own output | RTX 3080, 2.25 h |
| v2.3: voice encoder | 3 000 | from zero, model frozen | LibriTTS-R, 50 k clips, 460 speakers (40 held out) | RTX 3080, 0.2 h |
Common settings: Adam, bf16, length-bucketed batches with fixed shapes (MIOpen re-plans every new shape), learning-rate warm-up, and gradient clipping at 5. The last v2-en-long run was stopped at 50 000 of 55 000 steps, because the validation loss had stopped moving (1.5717 at 40 k, 1.5685 at 50 k). The new v2.1 parts start from zero, so the model loads the 50 k weights and sounds exactly the same at step 0 (checked: identical output).
Evaluation
Each figure is over the same 30 English sentences in 3 voices (90 clips), with the default delivery of each system:
- Real word errors: Whisper small.en reads the audio back, and we leave out what the scorer gets wrong for every system ("seashells" for "sea shells", "14" for "14th").
- PESQ, SI-SDR: no-reference estimates from torchaudio SQUIM (higher is better).
- Pace, pauses, pitch spread: pace is with leading and trailing silence trimmed; pauses are the share of the speech under the silence threshold; pitch spread is the standard deviation of the pitch in semitones.
| System | Real word errors | PESQ | SI-SDR | s / word | Pauses | Pitch spread |
|---|---|---|---|---|---|---|
| v2.1, studio voice 92, default pipeline (30 clips) | 2 / 420 | 3.90 | 27.2 | 0.360 | 16.5 % | 2.54 st |
| v2.1, three studio voices, no post-processing | 4 / 1 260 | 3.85 | 24.8 | 0.347 | 12.2 % | 2.74 st |
| v2.1, LibriTTS-R voices (pace +4 %, pauses x2.5) | 2 / 1 260 | 3.81 | 25.9 | 0.336 | 7.7 % | 2.63 st |
| v2.1, untouched durations | 5 / 1 260 | 3.82 | 25.6 | 0.306 | 5.5 % | 2.66 st |
| v2, 50 k steps, hand-set pace | 5 / 1 260 | 3.79 | 25.8 | 0.328 | 8.4 % | 2.89 st |
| v2, 25 k steps, 112 M BigVGAN-v2 | – | 2.41 | 14.2 | – | – | – |
The figures above were measured before the sentence_gap cap (sentence ends were stretched like commas then), so
the pace and pause share of the current default are a little faster and lower. The default studio voice reads slowly
on its own; pause_scale=1.4 shortens the pauses (13 % pause share) and length_scale=0.95 speeds it up.
What we learned on the way
- The big vocoder was the noise. On real mels the 112 M BigVGAN-v2 is the more faithful one (PESQ 4.08 against 3.80). On the mels the model generates it reproduces their fine jitter as hiss. The 14 M BigVGAN base, with no retraining, raised PESQ from 2.41 to 3.76 and is 8x smaller.
- Use the averaged weights. Benchmarks and demos must load the EMA weights, not the raw ones. The raw weights were measurably noisier.
- Regressed durations run short. Predicting log-durations biases speech fast and pauses short (see Architecture); sampled durations fix the bias at its source.
- Not every measured flaw needed fixing. The decoder is barely trained at flow time 0, where sampling starts (loss 0.172 against 0.129 at t = 0.01). Starting the solver at 0.01 changed nothing measurable, so the fix stays in training only.
- Per-band normalization is not free. Rewriting the 50 k weights for per-band mel normalization (exact for the prior and the output layer) still dropped PESQ from 3.79 to 2.95 before any fine-tuning. It needs its own run, and this model does not use it.
- Automatic scores saturate. Most remaining "word errors" were the scorer's, and PESQ cannot hear what a listener hears, as the status note above says.
What is next
- Emotions, as more style vectors next to "conversational".
- More conversational data from many speakers, so the style is less one pair of actors.
- A CPU vocoder that sounds as clean as BigVGAN (a larger or longer-trained Vocos).
pip install(the package is built, not yet published) and a smaller model for the browser demo.- Closer voice cloning: training the acoustic model together with the voice encoder.
- Other languages, starting with Slovak.
Limitations
- English only. Two styles, reading and conversation; no emotions yet. The conversational style comes from two actors, so it is one particular way of talking, not conversation in general.
- g2p_en handles common words well; rare names and words it does not know can come out wrong. The number rules are American English and read a bare four-digit number from 1100 to 2099 as a year ("1500 people" is "fifteen hundred").
- The voices belong to real LibriTTS-R and HiFi-TTS readers. Do not use them to impersonate anyone or to deceive.
- Speech is not watermarked.
Licenses and attribution
- This model (weights and code in this repo): CC BY-SA 4.0, because v2.2 learned from DailyTalk, which is CC BY-SA 4.0. Version v2.1 (in this repo's history, without styles) was trained only on CC BY data and stays CC BY 4.0.
- Training data: LibriTTS-R (Koizumi et al., 2023; derived from LibriTTS and LibriSpeech, CC BY 4.0), HiFi-TTS (Bakhturina et al., 2021, CC BY 4.0) and DailyTalk (Lee et al., "DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speech", ICASSP 2023, CC BY-SA 4.0).
- Vocoders: NVIDIA BigVGAN (Lee et al., 2023), MIT: the code comes from its repository, and
vocoder/bigvgan_finetuned.ptisnvidia/bigvgan_base_24khz_100bandfine-tuned here. Vocos (Siuzdak, "Vocos: Closing the gap between time-domain and Fourier-based neural vocoders", 2023), MIT:vocoder/vocos.safetensorsischaractr/vocos-mel-24khzfine-tuned here;tts_english/vocos.pyre-implements its forward pass. - G2P: g2p_en (Kyubyong Park), Apache-2.0, with the CMU Pronouncing Dictionary.
- Downloads last month
- 135