README.md

---
language:
- hi
- en
tags:
- text-to-speech
- tts
- flow-matching
- diffusion-transformer
- voice-cloning
- zero-shot
- elevenlabs-style
- pytorch
license: apache-2.0
pipeline_tag: text-to-speech
---

# 🎙️ ViuAI_TTS_200M: Next-Gen 2026 Speech Generation Model (~190.5M Parameters)

**ViuAI_TTS_200M** is a state-of-the-art (SOTA) Text-to-Speech deep learning architecture designed for human-grade speech synthesis and zero-shot voice cloning.

Inspired by modern breakthrough architectures (Flow Matching, DiT, ElevenLabs Adam, and Grok Voice), ViuAI_TTS_200M eliminates robotic artifacts through:
* **Optimal Transport Continuous Normalizing Flows (OT-CFM):** Generates crisp, clean speech in only 10–15 Euler ODE steps.
* **Diffusion Transformer (DiT) Backbone:** 9 deep DiT layers (~148.7M params) equipped with **AdaLN-Zero** and multi-head cross-attention.
* **Human Prosody Engine ($F_0$ Pitch + Energy + Duration):** Prevents robotic flat pitch by modeling dynamic intonation curves, emotional inflection, and natural micro-breaths.
* **In-Context Reference Voice Cloning:** Feed a 3–5 second human audio clip (`--ref_audio sample.wav`) to replicate any target voice's timbre, warmth, and cadence.
* **Classifier-Free Guidance (CFG Scale = 2.0x):** Delivers clean studio broadcast clarity.
* **24,000 Hz Neural Vocoder:** Multi-receptive field periodic generator producing uncompressed 24kHz 16-bit audio.

---

## 📊 Architecture & Parameter Budget Breakdown

| Component | Architecture Specs | Trainable Parameters |
| :--- | :--- | :--- |
| **Text & Prosody Encoder** | Pre-LN Transformer + RoPE (Multilingual) | **19.43 Million** |
| **Human Prosody Engine** | Dynamic $F_0$ Pitch + Energy + Duration ConvNeXt | **8.42 Million** |
| **DiT Core Backbone** | 9-layer Diffusion Transformer + AdaLN-Zero + CFG | **148.71 Million** |
| **Neural Audio Vocoder** | 24,000 Hz Multi-Receptive Periodic Generator | **13.93 Million** |
| **Total Model Parameters** | | **190.49 Million (~190.5M)** |

---

## 🚀 Quickstart & Inference

### 1. Zero-Shot Human Voice Cloning (ElevenLabs / Adam Style)
Clone any human voice by supplying a 3–5 second `.wav` audio prompt:

```bash
python inference.py \
  --text "Hello! This is ViuAI_TTS_200M speaking in a human-like voice." \
  --ref_audio "path/to/reference_sample.wav" \
  --cfg_scale 2.0 \
  --steps 15 \
  --output "cloned_output.wav"
```

### 2. Standard Text-to-Speech
```bash
python inference.py \
  --text "Namaste! Yeh ViuAI TTS 200M model ka audio output hai." \
  --cfg_scale 2.0 \
  --steps 15 \
  --output "speech.wav"
```

---

## 🏋️ Training Pipeline

Train with Mixed Precision (AMP), AdamW, and Cosine Annealing:
```bash
python train.py
```

Checkpoints are automatically saved to `checkpoints/` and can be loaded directly for inference.

---

## 📂 Project Repository Structure
```
ViuAI_TTS_200M/
├── config/
│   └── model_config.py          # 190.5M architectural hyperparameters
├── models/
│   ├── text_encoder.py          # Multilingual Transformer with RoPE
│   ├── dit_backbone.py          # Diffusion Transformer with Voice Conditioning
│   ├── duration_predictor.py    # Human Prosody Engine (F0, Energy, Duration)
│   ├── flow_matching.py         # Optimal Transport CFM & Euler ODE Solver
│   ├── vocoder.py               # 24kHz Neural Audio Vocoder
│   └── viuai_tts.py             # Unified ViuAITTS200M Model
├── dataset/
│   ├── audio_processing.py      # Mel-spectrogram extraction
│   └── tokenizer.py             # Hindi & English multilingual tokenizer
├── checkpoints/
│   └── viuai_tts_200m_init.pt   # Pretrained / initialized model weights
├── train.py                     # Mixed Precision (AMP) training script
├── inference.py                 # Text-to-speech audio generator
├── verify_model.py              # Architecture verification script
└── ROADMAP.md                   # Master engineering roadmap
```

---

## 📜 License
Apache 2.0