🎙️ ViuAI_TTS_200M: Next-Gen 2026 Speech Generation Model (~190.5M Parameters)

ViuAI_TTS_200M is a state-of-the-art (SOTA) Text-to-Speech deep learning architecture designed for human-grade speech synthesis and zero-shot voice cloning.

Inspired by modern breakthrough architectures (Flow Matching, DiT, ElevenLabs Adam, and Grok Voice), ViuAI_TTS_200M eliminates robotic artifacts through:

Optimal Transport Continuous Normalizing Flows (OT-CFM): Generates crisp, clean speech in only 10–15 Euler ODE steps.
Diffusion Transformer (DiT) Backbone: 9 deep DiT layers (~148.7M params) equipped with AdaLN-Zero and multi-head cross-attention.
Human Prosody Engine ($F_0$ Pitch + Energy + Duration): Prevents robotic flat pitch by modeling dynamic intonation curves, emotional inflection, and natural micro-breaths.
In-Context Reference Voice Cloning: Feed a 3–5 second human audio clip (--ref_audio sample.wav) to replicate any target voice's timbre, warmth, and cadence.
Classifier-Free Guidance (CFG Scale = 2.0x): Delivers clean studio broadcast clarity.
24,000 Hz Neural Vocoder: Multi-receptive field periodic generator producing uncompressed 24kHz 16-bit audio.

📊 Architecture & Parameter Budget Breakdown

Component	Architecture Specs	Trainable Parameters
Text & Prosody Encoder	Pre-LN Transformer + RoPE (Multilingual)	19.43 Million
Human Prosody Engine	Dynamic $F_0$ Pitch + Energy + Duration ConvNeXt	8.42 Million
DiT Core Backbone	9-layer Diffusion Transformer + AdaLN-Zero + CFG	148.71 Million
Neural Audio Vocoder	24,000 Hz Multi-Receptive Periodic Generator	13.93 Million
Total Model Parameters		190.49 Million (~190.5M)

🚀 Quickstart & Inference

1. Zero-Shot Human Voice Cloning (ElevenLabs / Adam Style)

Clone any human voice by supplying a 3–5 second .wav audio prompt:

python inference.py \
  --text "Hello! This is ViuAI_TTS_200M speaking in a human-like voice." \
  --ref_audio "path/to/reference_sample.wav" \
  --cfg_scale 2.0 \
  --steps 15 \
  --output "cloned_output.wav"

2. Standard Text-to-Speech

python inference.py \
  --text "Namaste! Yeh ViuAI TTS 200M model ka audio output hai." \
  --cfg_scale 2.0 \
  --steps 15 \
  --output "speech.wav"

🏋️ Training Pipeline

Train with Mixed Precision (AMP), AdamW, and Cosine Annealing:

python train.py

Checkpoints are automatically saved to checkpoints/ and can be loaded directly for inference.

📂 Project Repository Structure

ViuAI_TTS_200M/
├── config/
│   └── model_config.py          # 190.5M architectural hyperparameters
├── models/
│   ├── text_encoder.py          # Multilingual Transformer with RoPE
│   ├── dit_backbone.py          # Diffusion Transformer with Voice Conditioning
│   ├── duration_predictor.py    # Human Prosody Engine (F0, Energy, Duration)
│   ├── flow_matching.py         # Optimal Transport CFM & Euler ODE Solver
│   ├── vocoder.py               # 24kHz Neural Audio Vocoder
│   └── viuai_tts.py             # Unified ViuAITTS200M Model
├── dataset/
│   ├── audio_processing.py      # Mel-spectrogram extraction
│   └── tokenizer.py             # Hindi & English multilingual tokenizer
├── checkpoints/
│   └── viuai_tts_200m_init.pt   # Pretrained / initialized model weights
├── train.py                     # Mixed Precision (AMP) training script
├── inference.py                 # Text-to-speech audio generator
├── verify_model.py              # Architecture verification script
└── ROADMAP.md                   # Master engineering roadmap

📜 License

Apache 2.0

Downloads last month: -; Downloads are not tracked for this model. How to track