Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,267 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<div align="center">
|
| 2 |
+
|
| 3 |
+
<!-- Laila Logo -->
|
| 4 |
+
<img src="https://github.com/user-attachments/assets/560b6d36-7aef-4d72-bcac-1171fcb9e8c9" alt="Laila Logo" width="300"/>
|
| 5 |
+
|
| 6 |
+
### Parameter-Efficient Adaptation of Whisper-Small for Indian-Accented and Code-Switched Hindi-English Speech
|
| 7 |
+
|
| 8 |
+
[](https://github.com/techindro)
|
| 9 |
+
[](https://opensource.org/licenses/MIT)
|
| 10 |
+
[](https://www.python.org/downloads/)
|
| 11 |
+
[](https://pytorch.org/)
|
| 12 |
+
[](https://huggingface.co/docs/transformers)
|
| 13 |
+
[](https://github.com/huggingface/peft)
|
| 14 |
+
[](https://neurips.cc)
|
| 15 |
+
|
| 16 |
+
**3.5M trainable params · 1.5% of base model · Trains on 1 free T4 GPU**
|
| 17 |
+
|
| 18 |
+
</div>
|
| 19 |
+
|
| 20 |
---
|
| 21 |
+
|
| 22 |
+
## 🌐 Platform Overview
|
| 23 |
+
|
| 24 |
+
> *"Whisper, but for India — parameter-efficient ASR for Indian-accented and code-switched speech"*
|
| 25 |
+
|
| 26 |
+
Laila provides high-quality speech recognition, model evaluation, and AI tools designed for Indian users, enterprises, and researchers working on next-generation voice applications.
|
| 27 |
+
|
| 28 |
---
|
| 29 |
+
|
| 30 |
+
## 🚀 Live Demos
|
| 31 |
+
|
| 32 |
+
Experience Laila in action. All demos run on Hugging Face Spaces — no installation required.
|
| 33 |
+
|
| 34 |
+
| Demo | Description | Link |
|
| 35 |
+
|------|-------------|------|
|
| 36 |
+
| 🎙️ Hindi ASR | Hindi speech recognition with LoRA-tuned Whisper | [Try Now](https://huggingface.co/spaces/techindro/Laila-Demo) |
|
| 37 |
+
| 🔀 Hinglish ASR | Code-switched Hindi-English transcription | [Try Now](https://huggingface.co/spaces/techindro/Laila-Hinglish) |
|
| 38 |
+
| 🇮🇳 Indian English ASR | Indian-accented English transcription | [Try Now](https://huggingface.co/spaces/techindro/Laila-IndianEnglish) |
|
| 39 |
+
| 🌐 22-Language Suite | One adapter per scheduled Indian language | [Coming Soon](https://huggingface.co/techindro) |
|
| 40 |
+
|
| 41 |
+
🌐 Repository: [github.com/techindro/Laila](https://github.com/techindro/Laila)
|
| 42 |
+
|
| 43 |
+
---
|
| 44 |
+
|
| 45 |
+
## 📖 What is Laila?
|
| 46 |
+
|
| 47 |
+
Laila is a **parameter-efficient speech recognition model** purpose-built for India's linguistic diversity. It adapts the base Whisper-Small model using LoRA, training only **1.5% of parameters** while keeping all 244M base weights frozen.
|
| 48 |
+
|
| 49 |
+
### The Name
|
| 50 |
+
|
| 51 |
+
- **Laila** = "Night" in Arabic/Persian, "play" in Hindi — evoking speech and expression
|
| 52 |
+
- **Model** = Speech recognition for Indian languages
|
| 53 |
+
|
| 54 |
+
So Laila means bringing clarity to Indian speech, day or night.
|
| 55 |
+
|
| 56 |
+
### What Problem Does It Solve?
|
| 57 |
+
|
| 58 |
+
Global speech models like Whisper, trained mostly on Western English, make more errors on Indian speech because:
|
| 59 |
+
|
| 60 |
+
1. Indian-accented English has different phonetics, rhythm, and stress patterns
|
| 61 |
+
2. Everyday speech is often code-switched (Hindi + English in one sentence)
|
| 62 |
+
3. No single accepted transcript convention exists (Devanagari vs. Roman)
|
| 63 |
+
4. Full fine-tuning is costly — needs more memory and produces full model copies
|
| 64 |
+
|
| 65 |
+
**The result:** Speech recognition that works perfectly in San Francisco but struggles in Mumbai, Delhi, and Chennai.
|
| 66 |
+
|
| 67 |
+
### How Does Laila Work?
|
| 68 |
+
|
| 69 |
+
The pipeline has five parts that work together:
|
| 70 |
+
|
| 71 |
+
| Part | What It Does |
|
| 72 |
+
|------|---------------|
|
| 73 |
+
| 🎧 Audio Preprocessing | Resampling, mono conversion, normalization, 30s chunking |
|
| 74 |
+
| 🧠 Whisper-Small | Frozen 244M-parameter encoder-decoder Transformer |
|
| 75 |
+
| 🔗 LoRA Adapters | Trains 3.5M params on `q_proj` & `v_proj` attention layers |
|
| 76 |
+
| 🔊 Noise Augmentation | Mixes real background noise at random SNRs (train only) |
|
| 77 |
+
| 📝 Transcript Convention | Devanagari for Hindi, Latin for English, normalized scoring |
|
| 78 |
+
|
| 79 |
+
---
|
| 80 |
+
|
| 81 |
+
## 📊 Key Results at a Glance
|
| 82 |
+
|
| 83 |
+
| Metric | Laila | Whisper-Small (zero-shot) | Improvement |
|
| 84 |
+
|--------|-------|---------------------------|-------------|
|
| 85 |
+
| Trainable Parameters | 3.5M | 244M | **1.5% of base** |
|
| 86 |
+
| Adapter Size | ~few MB | 244M | **~70× smaller** |
|
| 87 |
+
| Hardware | 1× T4 (16GB) | 1× T4 (16GB) | **Free-tier** |
|
| 88 |
+
| Hindi WER | TBD | TBD | TBD |
|
| 89 |
+
| Indian-English WER | TBD | TBD | TBD |
|
| 90 |
+
| Code-Switched WER | TBD | TBD | TBD |
|
| 91 |
+
|
| 92 |
+
---
|
| 93 |
+
|
| 94 |
+
## 🎯 The Problem
|
| 95 |
+
|
| 96 |
+
**Global ASR training data ignores 1.4 billion Indian voices.**
|
| 97 |
+
|
| 98 |
+
Existing models like Whisper were trained on 680k hours of mostly English data:
|
| 99 |
+
|
| 100 |
+
| Limitation | Consequence |
|
| 101 |
+
|------------|-------------|
|
| 102 |
+
| Western English bias | Errors on Indian-accented English |
|
| 103 |
+
| Monolingual training | Fails on code-switched speech |
|
| 104 |
+
| Mixed-script ambiguity | Inconsistent training/evaluation |
|
| 105 |
+
| Full fine-tuning cost | High memory, full model copies per variant |
|
| 106 |
+
|
| 107 |
+
---
|
| 108 |
+
|
| 109 |
+
## 🚀 The Solution
|
| 110 |
+
|
| 111 |
+
Laila is a **parameter-efficient adaptation** of Whisper-Small purpose-built for India's linguistic diversity.
|
| 112 |
+
|
| 113 |
+
### Comparison with Baselines
|
| 114 |
+
|
| 115 |
+
| Feature | Whisper-Small (zero-shot) | Whisper-Small (full FT) | Laila (LoRA) |
|
| 116 |
+
|---------|---------------------------|--------------------------|--------------|
|
| 117 |
+
| Hindi Support | Partial | ✅ Yes | ✅ Yes |
|
| 118 |
+
| Indian-Accented English | Partial | ✅ Yes | ✅ Yes |
|
| 119 |
+
| Code-Switched (Hinglish) | ❌ | ✅ Yes | ✅ Yes |
|
| 120 |
+
| Trainable Params | 0 | 244M | **3.5M** |
|
| 121 |
+
| Adapter Size | — | Full model | **Few MB** |
|
| 122 |
+
| Memory (Training) | — | High | **16GB T4** |
|
| 123 |
+
| Base Model Weights | Frozen | Fully trained | **Frozen** |
|
| 124 |
+
|
| 125 |
+
---
|
| 126 |
+
|
| 127 |
+
## 🧠 AI Knowledge Hub
|
| 128 |
+
|
| 129 |
+
Laila explores core pillars of modern speech AI and how LoRA applies each discipline to solve critical language challenges.
|
| 130 |
+
|
| 131 |
+
### Core AI Disciplines
|
| 132 |
+
|
| 133 |
+
**Parameter-Efficient Fine-Tuning** — *Train less. Do more.*
|
| 134 |
+
> LoRA writes the update to a weight matrix as the product of two low-rank matrices and trains only these. This lowers memory use and the size of the stored update. Laila leverages LoRA to adapt Whisper-Small with just 1.5% of trainable parameters.
|
| 135 |
+
- 3.5M trainable params vs. 244M base
|
| 136 |
+
- Single free-tier T4 GPU
|
| 137 |
+
- Adapter file of a few MB
|
| 138 |
+
|
| 139 |
+
**Code-Switched Speech** — *Mixed language. Unified model.*
|
| 140 |
+
> Code-switched speech mixes languages within a sentence — e.g., "kal meeting ke baad report bhej dena." Laila handles Hindi grammar with English nouns using separate test sets and a fixed transcript convention.
|
| 141 |
+
- Separate Hindi, English, and code-switched test sets
|
| 142 |
+
- Devanagari + Latin script convention
|
| 143 |
+
- Noise-robust training
|
| 144 |
+
|
| 145 |
+
---
|
| 146 |
+
|
| 147 |
+
## 📂 Case Studies & Resources
|
| 148 |
+
|
| 149 |
+
### Highlighted Case Studies
|
| 150 |
+
|
| 151 |
+
| Case Study | Category | Description |
|
| 152 |
+
|------------|----------|-------------|
|
| 153 |
+
| **Hindi ASR with LoRA** | ASR | Adapting Whisper-Small for Hindi speech on a single free T4 GPU |
|
| 154 |
+
| **Hinglish Code-Switching** | Code-Switching | Building a transcription pipeline for Hindi-English mixed speech |
|
| 155 |
+
| **Noise-Robust ASR** | Robustness | Training with background noise for real-world conditions |
|
| 156 |
+
|
| 157 |
+
### Learning Resources
|
| 158 |
+
|
| 159 |
+
| Type | Title |
|
| 160 |
+
|------|-------|
|
| 161 |
+
| Paper | Laila: Parameter-Efficient Adaptation of Whisper-Small |
|
| 162 |
+
| Guide | Guide to LoRA Fine-Tuning |
|
| 163 |
+
| Guide | Guide: Whisper Architecture |
|
| 164 |
+
| Guide | Guide: WER & CER Evaluation |
|
| 165 |
+
|
| 166 |
+
---
|
| 167 |
+
|
| 168 |
+
## 📊 Benchmark Results
|
| 169 |
+
|
| 170 |
+
### Word Error Rate (WER, %)
|
| 171 |
+
|
| 172 |
+
| Model | Hindi | Indian English | Code-Switched | Trainable Params |
|
| 173 |
+
|-------|-------|----------------|---------------|------------------|
|
| 174 |
+
| Whisper-Small (zero-shot) | TBD | TBD | TBD | 0 |
|
| 175 |
+
| Whisper-Small (full fine-tune) | TBD | TBD | TBD | 244M |
|
| 176 |
+
| **Laila (LoRA, r=32)** | **TBD** | **TBD** | **TBD** | **3.5M** |
|
| 177 |
+
|
| 178 |
+
### Character Error Rate (CER, %)
|
| 179 |
+
|
| 180 |
+
| Model | Hindi | Indian English | Code-Switched |
|
| 181 |
+
|-------|-------|----------------|---------------|
|
| 182 |
+
| Whisper-Small (zero-shot) | TBD | TBD | TBD |
|
| 183 |
+
| Whisper-Small (full fine-tune) | TBD | TBD | TBD |
|
| 184 |
+
| **Laila (LoRA, r=32)** | **TBD** | **TBD** | **TBD** |
|
| 185 |
+
|
| 186 |
+
### Ablation: LoRA Rank
|
| 187 |
+
|
| 188 |
+
| Rank | Trainable Params | WER (%) |
|
| 189 |
+
|------|------------------|---------|
|
| 190 |
+
| 8 | TBD | TBD |
|
| 191 |
+
| 16 | TBD | TBD |
|
| 192 |
+
| **32** | **3.5M** | **TBD** |
|
| 193 |
+
| 64 | TBD | TBD |
|
| 194 |
+
|
| 195 |
+
### Ablation: Adapted Modules & Noise Augmentation
|
| 196 |
+
|
| 197 |
+
| Adapted Modules | Noise | Hindi | English | Code-Switched |
|
| 198 |
+
|-----------------|-------|-------|---------|---------------|
|
| 199 |
+
| `q_proj`, `v_proj` | Off | TBD | TBD | TBD |
|
| 200 |
+
| `q_proj`, `v_proj` | On | TBD | TBD | TBD |
|
| 201 |
+
| `q_proj`, `k_proj`, `v_proj`, `out_proj` | On | TBD | TBD | TBD |
|
| 202 |
+
| Decoder only | On | TBD | TBD | TBD |
|
| 203 |
+
|
| 204 |
+
---
|
| 205 |
+
|
| 206 |
+
## 🎙️ Indian Language Support (22 Adapters Planned)
|
| 207 |
+
|
| 208 |
+
Laila plans to support all 22 scheduled languages of India:
|
| 209 |
+
|
| 210 |
+
| Category | Languages |
|
| 211 |
+
|----------|-----------|
|
| 212 |
+
| **Indo-Aryan** | Hindi (हिन्दी), Bengali (বাংলা), Marathi (मराठी), Gujarati (ગુજરાતી), Odia (ଓଡ଼ିଆ), Punjabi (ਪੰਜਾਬੀ), Assamese (অসমীয়া), Urdu (اردو), Konkani (कोंकणी), Nepali (नेपाली), Maithili (मैथिली), Dogri (डोगरी), Sindhi (سنڌي) |
|
| 213 |
+
| **Dravidian** | Tamil (தமிழ்), Telugu (తెలుగు), Kannada (ಕನ್ನಡ), Malayalam (മലയാളം) |
|
| 214 |
+
| **Other** | Sanskrit (संस्कृतम्), Manipuri (মণিপুরী), Bodo (बड़ो), Santali (ᱥᱟᱱᱛᱟᱲᱤ), Kashmiri (کٲشُر) |
|
| 215 |
+
| **Code-Switching** | Hinglish, Tanglish, Benglish, Tinglish, Kanglish, Manglish, Punglish, Marathi-English |
|
| 216 |
+
|
| 217 |
+
---
|
| 218 |
+
|
| 219 |
+
## 📁 Dataset v1.0 Statistics
|
| 220 |
+
|
| 221 |
+
| Split | Modality | Samples | Annotated Labels |
|
| 222 |
+
|-------|----------|---------|------------------|
|
| 223 |
+
| Train | Hindi (Common Voice + AI4Bharat) | TBD | TBD |
|
| 224 |
+
| Val | Hindi | TBD | TBD |
|
| 225 |
+
| Train | Indian-Accented English | TBD | TBD |
|
| 226 |
+
| Val | Indian-Accented English | TBD | TBD |
|
| 227 |
+
| Train | Code-Switched Hindi-English | TBD | TBD |
|
| 228 |
+
| Val | Code-Switched Hindi-English | TBD | TBD |
|
| 229 |
+
| **Total** | **All** | **TBD** | **TBD** |
|
| 230 |
+
|
| 231 |
+
---
|
| 232 |
+
|
| 233 |
+
## 🏗️ Technology Stack
|
| 234 |
+
|
| 235 |
+
| Layer | Technologies |
|
| 236 |
+
|-------|--------------|
|
| 237 |
+
| Base Model | Whisper-Small (244M parameters) |
|
| 238 |
+
| Adaptation | LoRA (PEFT), rank r=32, α=64, dropout=0.05 |
|
| 239 |
+
| Deep Learning | PyTorch 2.0+, HuggingFace Transformers |
|
| 240 |
+
| Audio Processing | torchaudio, librosa, soundfile |
|
| 241 |
+
| Evaluation | jiwer (WER/CER), HuggingFace Evaluate |
|
| 242 |
+
| Training | HuggingFace Trainer, FP16, gradient checkpointing |
|
| 243 |
+
| Inference | PEFT + Transformers, serverless worker architecture |
|
| 244 |
+
| Infrastructure | 1× free-tier T4 GPU (16GB) |
|
| 245 |
+
|
| 246 |
+
---
|
| 247 |
+
|
| 248 |
+
## 🔮 What's Next?
|
| 249 |
+
|
| 250 |
+
- Support for all 22 Indian languages
|
| 251 |
+
- Real-time speech data processing
|
| 252 |
+
- API for companies to use
|
| 253 |
+
- Expansion to other code-switched pairs
|
| 254 |
+
- Deeper integration with AI4Bharat and Bhashini
|
| 255 |
+
- Expansion of noise-robust ASR for defense applications
|
| 256 |
+
|
| 257 |
+
---
|
| 258 |
+
|
| 259 |
+
## 📜 License
|
| 260 |
+
|
| 261 |
+
MIT — Free to use, modify, and distribute.
|
| 262 |
+
|
| 263 |
+
---
|
| 264 |
+
|
| 265 |
+
## 🤝 Contributing
|
| 266 |
+
|
| 267 |
+
PRs welcome! Let's build speech AI for Bharat 🇮🇳
|