nitrai-research/ModeratoTTS-V1.1-Preview

ModeratoTTS V1.1 Preview — Two-Tier Proodic TTS (49.7M, FP8)

A from-scratch ~50M-parameter text-to-speech model with voice cloning, trained on real conversational speech from the Open-Yap 1K sample (CC-BY-4.0; 16 conversations, 8 speakers, word-level timestamps). Published as-is

Model

• 49.73M params: Director 9.07M + Acoustic Core 38.34M + Vocoder 1.58M + voice system 0.75M
• FP8 (float8_e4m3fn) heavy linears via scaled-mm, FP32 master weights; trained on a single H100
• 6 built-in voices (3 female / 3 male): alice, bella, cora, david, eric, fred + zero-shot voice cloning from a reference clip
• 22.05 kHz, 80-bin log-mel; word-timestamp alignments used for training

Training summary

• Data: 4,514 utterances / 8.07 h segmented from Open-Yap by word gaps
• 110,000 steps total (mel stage 20k + GAN stage 8k + mel resume 82k), batch 16, ~8 steps/s
• Final metrics: loss/mel 0.30, loss/duration 0.16, speaker/acc 1.00, vad/acc 0.82
• Vocoder: trained separately on real-mel crops with multi-resolution STFT loss (12k steps) — see tools/train_vocoder.py in the source repo

Usage

pip install -r requirements.txt   # torch 2.9.1+cu128, numpy, soundfile
python synthesize.py --checkpoint model.pt --list-voices
python synthesize.py --checkpoint model.pt --text "Hello world" --voice david --out david.wav
python synthesize.py --checkpoint model.pt --text "Hello" --reference-wav ref.wav --out clone.wav

Listen to samples/*.wav (note the hallucination limitation above).

Data attribution

Trained on TheAgenticDataCompany/open-yap-1k sample (CC-BY-4.0). Weights in this preview are released under CC-BY-4.0 with attribution to the Open-Yap dataset creators. Do not use cloned voices to impersonate real people.