Add model card

README.md

@@ -0,0 +1,192 @@
+---
+license: other
+language:
+- bn
+tags:
+- text-to-speech
+- bangla
+- xtts
+- coqui
+- tts
+- fine-tuned
+- checkpoints
+library_name: coqui
+---
+
+# XTTS v2 Bangla — Training Checkpoints
+
+# **Zero-shot voice cloning — provide just 5 seconds of any speaker's audio and the model will synthesize Bangla speech in that voice. No retraining required.**
+
+This repository contains **5 training checkpoints** from the XTTS v2 Bangla fine-tuning run,
+evenly spaced across steps 382,000–406,000.
+
+For the **production-ready best model** (step 406,521), see:
+👉 [EMTIAZZ/xtts-bangla-finetuned](https://huggingface.co/EMTIAZZ/xtts-bangla-finetuned)
+
+---
+
+## Checkpoints
+
+| File | Step | Notes |
+|------|------|-------|
+| `checkpoint_382000.pth` | 382,000 | Earliest checkpoint in this set |
+| `checkpoint_389000.pth` | 389,000 | |
+| `checkpoint_395000.pth` | 395,000 | |
+| `checkpoint_401000.pth` | 401,000 | |
+| `checkpoint_406000.pth` | 406,000 | Latest regular checkpoint |
+
+> These are full model checkpoints ready for inference (GPT + HiFi-GAN + mel stats).
+> Use `checkpoint_406000.pth` for best results, or compare earlier steps to study the training progression.
+
+---
+
+## Quick Start
+
+### 1. Install dependencies
+
+```bash
+pip install "TTS==0.22.0" "transformers==4.36.0" "tokenizers==0.15.0" soundfile
+```
+
+> **Important:** Use these exact versions. Newer transformers versions break XTTS inference.
+
+### 2. Run inference
+
+```bash
+python inference.py --text "আমি বাংলায় কথা বলতে পারি।" --checkpoint checkpoint_406000.pth --output output.wav
+```
+
+**Clone any voice with just 5 seconds of audio:**
+
+```bash
+python inference.py \
+    --text "আমি বাংলায় কথা বলতে পারি।" \
+    --checkpoint checkpoint_406000.pth \
+    --ref_audio your_speaker.wav \
+    --output output.wav
+```
+
+> The `--ref_audio` file can be as short as **5 seconds**. The model extracts a speaker
+> embedding from it and generates speech in that voice. WAV or MP3 works.
+
+---
+
+## Python API
+
+```python
+import soundfile as sf
+from huggingface_hub import snapshot_download
+from TTS.tts.configs.xtts_config import XttsConfig
+from TTS.tts.models.xtts import Xtts
+
+# Download all files (~26 GB total)
+model_dir = snapshot_download("EMTIAZZ/xtts-bangla-checkpoints")
+
+# Choose a checkpoint — use checkpoint_406000.pth for best results
+checkpoint_path = f"{model_dir}/checkpoint_406000.pth"
+
+# Load model
+config = XttsConfig()
+config.load_json(f"{model_dir}/config.json")
+model = Xtts.init_from_config(config)
+model.load_checkpoint(
+    config,
+    checkpoint_dir=model_dir,
+    checkpoint_path=checkpoint_path,
+    vocab_path=f"{model_dir}/vocab.json",
+    use_deepspeed=False,
+    eval=True,
+)
+model.cpu()   # or model.cuda() on GPU
+
+# Speaker conditioning — just 5 seconds of audio is enough
+gpt_cond_latent, speaker_embedding = model.get_conditioning_latents(
+    audio_path=f"{model_dir}/reference_audio.wav",
+    gpt_cond_len=config.gpt_cond_len,
+    max_ref_length=config.max_ref_len,
+    sound_norm_refs=config.sound_norm_refs,
+)
+
+# Generate speech — use language="en" (NOT "bn")
+outputs = model.inference(
+    text="আমি বাংলায় কথা বলতে পারি।",
+    language="en",
+    gpt_cond_latent=gpt_cond_latent,
+    speaker_embedding=speaker_embedding,
+    temperature=0.1,
+    length_penalty=1.0,
+    repetition_penalty=10.0,
+    top_k=10,
+    top_p=0.3,
+)
+sf.write("output.wav", outputs["wav"], 24000)
+```
+
+---
+
+## Important: Why `language="en"`?
+
+This model was fine-tuned using `language="en"` to bypass XTTS's language-specific
+tokenizer preprocessing. The Bangla text is encoded directly as Unicode characters.
+
+**Using `language="bn"` will produce garbled/wrong audio.**
+Always use `language="en"` when running inference, even for pure Bangla text.
+
+---
+
+## Inference Parameters (Production)
+
+| Parameter | Value | Notes |
+|-----------|-------|-------|
+| `temperature` | `0.1` | Low = more deterministic |
+| `length_penalty` | `1.0` | Default |
+| `repetition_penalty` | `10.0` | Prevents repetition/loops |
+| `top_k` | `10` | |
+| `top_p` | `0.3` | |
+| `language` | `"en"` | **Required** |
+
+---
+
+## Files
+
+| File | Size | Description |
+|------|------|-------------|
+| `checkpoint_382000.pth` | 5.3 GB | Step 382,000 |
+| `checkpoint_389000.pth` | 5.3 GB | Step 389,000 |
+| `checkpoint_395000.pth` | 5.3 GB | Step 395,000 |
+| `checkpoint_401000.pth` | 5.3 GB | Step 401,000 |
+| `checkpoint_406000.pth` | 5.3 GB | Step 406,000 |
+| `config.json` | 5 KB | Model configuration |
+| `vocab.json` | 331 KB | BPE vocabulary (8,565 Bangla tokens) |
+| `reference_audio.wav` | 295 KB | Sample reference speaker audio (Adiba voice) |
+| `inference.py` | — | Ready-to-run inference script |
+
+---
+
+## Training Details
+
+| | |
+|---|---|
+| **Base model** | Coqui XTTS v2 (`tts_models/multilingual/multi-dataset/xtts_v2`) |
+| **Dataset** | ~99K Bangla speech samples |
+| **Training platform** | Modal (H100 GPU) |
+| **Fine-tuning date** | January 2026 |
+| **Vocabulary** | 8,565 BPE tokens (Bangla-extended) |
+| **Output sample rate** | 24,000 Hz |
+
+---
+
+## Troubleshooting
+
+**`ImportError` from transformers / tokenizers mismatch**
+Install exact versions: `pip install "transformers==4.36.0" "tokenizers==0.15.0"`
+
+**Audio is garbled or very short**
+Make sure you are using `language="en"`, not `"bn"`.
+
+**Slow inference on CPU**
+XTTS has known MPS issues on Apple Silicon. CPU is the safest option.
+On Linux/Windows with NVIDIA GPU, use `model.cuda()` for fast inference.
+
+**Out of memory**
+Each checkpoint is ~5.3 GB. On CPU you need ~6 GB RAM. On GPU you need ~6 GB VRAM.