techindro commited on
Commit
1c36f19
·
verified ·
1 Parent(s): 91218a5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +265 -1
README.md CHANGED
@@ -1,3 +1,267 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
- license: apache-2.0
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <div align="center">
2
+
3
+ <!-- Laila Logo -->
4
+ <img src="https://github.com/user-attachments/assets/560b6d36-7aef-4d72-bcac-1171fcb9e8c9" alt="Laila Logo" width="300"/>
5
+
6
+ ### Parameter-Efficient Adaptation of Whisper-Small for Indian-Accented and Code-Switched Hindi-English Speech
7
+
8
+ [![GitHub](https://img.shields.io/badge/GitHub-techindro-black?logo=github)](https://github.com/techindro)
9
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
10
+ [![Python 3.10+](https://img.shields.io/badge/Python-3.10+-blue?logo=python)](https://www.python.org/downloads/)
11
+ [![PyTorch](https://img.shields.io/badge/PyTorch-2.0+-red?logo=pytorch)](https://pytorch.org/)
12
+ [![Hugging Face](https://img.shields.io/badge/🤗-Transformers-orange)](https://huggingface.co/docs/transformers)
13
+ [![PEFT](https://img.shields.io/badge/PEFT-LoRA-purple)](https://github.com/huggingface/peft)
14
+ [![NeurIPS](https://img.shields.io/badge/NeurIPS-2026-blueviolet)](https://neurips.cc)
15
+
16
+ **3.5M trainable params · 1.5% of base model · Trains on 1 free T4 GPU**
17
+
18
+ </div>
19
+
20
  ---
21
+
22
+ ## 🌐 Platform Overview
23
+
24
+ > *"Whisper, but for India — parameter-efficient ASR for Indian-accented and code-switched speech"*
25
+
26
+ Laila provides high-quality speech recognition, model evaluation, and AI tools designed for Indian users, enterprises, and researchers working on next-generation voice applications.
27
+
28
  ---
29
+
30
+ ## 🚀 Live Demos
31
+
32
+ Experience Laila in action. All demos run on Hugging Face Spaces — no installation required.
33
+
34
+ | Demo | Description | Link |
35
+ |------|-------------|------|
36
+ | 🎙️ Hindi ASR | Hindi speech recognition with LoRA-tuned Whisper | [Try Now](https://huggingface.co/spaces/techindro/Laila-Demo) |
37
+ | 🔀 Hinglish ASR | Code-switched Hindi-English transcription | [Try Now](https://huggingface.co/spaces/techindro/Laila-Hinglish) |
38
+ | 🇮🇳 Indian English ASR | Indian-accented English transcription | [Try Now](https://huggingface.co/spaces/techindro/Laila-IndianEnglish) |
39
+ | 🌐 22-Language Suite | One adapter per scheduled Indian language | [Coming Soon](https://huggingface.co/techindro) |
40
+
41
+ 🌐 Repository: [github.com/techindro/Laila](https://github.com/techindro/Laila)
42
+
43
+ ---
44
+
45
+ ## 📖 What is Laila?
46
+
47
+ Laila is a **parameter-efficient speech recognition model** purpose-built for India's linguistic diversity. It adapts the base Whisper-Small model using LoRA, training only **1.5% of parameters** while keeping all 244M base weights frozen.
48
+
49
+ ### The Name
50
+
51
+ - **Laila** = "Night" in Arabic/Persian, "play" in Hindi — evoking speech and expression
52
+ - **Model** = Speech recognition for Indian languages
53
+
54
+ So Laila means bringing clarity to Indian speech, day or night.
55
+
56
+ ### What Problem Does It Solve?
57
+
58
+ Global speech models like Whisper, trained mostly on Western English, make more errors on Indian speech because:
59
+
60
+ 1. Indian-accented English has different phonetics, rhythm, and stress patterns
61
+ 2. Everyday speech is often code-switched (Hindi + English in one sentence)
62
+ 3. No single accepted transcript convention exists (Devanagari vs. Roman)
63
+ 4. Full fine-tuning is costly — needs more memory and produces full model copies
64
+
65
+ **The result:** Speech recognition that works perfectly in San Francisco but struggles in Mumbai, Delhi, and Chennai.
66
+
67
+ ### How Does Laila Work?
68
+
69
+ The pipeline has five parts that work together:
70
+
71
+ | Part | What It Does |
72
+ |------|---------------|
73
+ | 🎧 Audio Preprocessing | Resampling, mono conversion, normalization, 30s chunking |
74
+ | 🧠 Whisper-Small | Frozen 244M-parameter encoder-decoder Transformer |
75
+ | 🔗 LoRA Adapters | Trains 3.5M params on `q_proj` & `v_proj` attention layers |
76
+ | 🔊 Noise Augmentation | Mixes real background noise at random SNRs (train only) |
77
+ | 📝 Transcript Convention | Devanagari for Hindi, Latin for English, normalized scoring |
78
+
79
+ ---
80
+
81
+ ## 📊 Key Results at a Glance
82
+
83
+ | Metric | Laila | Whisper-Small (zero-shot) | Improvement |
84
+ |--------|-------|---------------------------|-------------|
85
+ | Trainable Parameters | 3.5M | 244M | **1.5% of base** |
86
+ | Adapter Size | ~few MB | 244M | **~70× smaller** |
87
+ | Hardware | 1× T4 (16GB) | 1× T4 (16GB) | **Free-tier** |
88
+ | Hindi WER | TBD | TBD | TBD |
89
+ | Indian-English WER | TBD | TBD | TBD |
90
+ | Code-Switched WER | TBD | TBD | TBD |
91
+
92
+ ---
93
+
94
+ ## 🎯 The Problem
95
+
96
+ **Global ASR training data ignores 1.4 billion Indian voices.**
97
+
98
+ Existing models like Whisper were trained on 680k hours of mostly English data:
99
+
100
+ | Limitation | Consequence |
101
+ |------------|-------------|
102
+ | Western English bias | Errors on Indian-accented English |
103
+ | Monolingual training | Fails on code-switched speech |
104
+ | Mixed-script ambiguity | Inconsistent training/evaluation |
105
+ | Full fine-tuning cost | High memory, full model copies per variant |
106
+
107
+ ---
108
+
109
+ ## 🚀 The Solution
110
+
111
+ Laila is a **parameter-efficient adaptation** of Whisper-Small purpose-built for India's linguistic diversity.
112
+
113
+ ### Comparison with Baselines
114
+
115
+ | Feature | Whisper-Small (zero-shot) | Whisper-Small (full FT) | Laila (LoRA) |
116
+ |---------|---------------------------|--------------------------|--------------|
117
+ | Hindi Support | Partial | ✅ Yes | ✅ Yes |
118
+ | Indian-Accented English | Partial | ✅ Yes | ✅ Yes |
119
+ | Code-Switched (Hinglish) | ❌ | ✅ Yes | ✅ Yes |
120
+ | Trainable Params | 0 | 244M | **3.5M** |
121
+ | Adapter Size | — | Full model | **Few MB** |
122
+ | Memory (Training) | — | High | **16GB T4** |
123
+ | Base Model Weights | Frozen | Fully trained | **Frozen** |
124
+
125
+ ---
126
+
127
+ ## 🧠 AI Knowledge Hub
128
+
129
+ Laila explores core pillars of modern speech AI and how LoRA applies each discipline to solve critical language challenges.
130
+
131
+ ### Core AI Disciplines
132
+
133
+ **Parameter-Efficient Fine-Tuning** — *Train less. Do more.*
134
+ > LoRA writes the update to a weight matrix as the product of two low-rank matrices and trains only these. This lowers memory use and the size of the stored update. Laila leverages LoRA to adapt Whisper-Small with just 1.5% of trainable parameters.
135
+ - 3.5M trainable params vs. 244M base
136
+ - Single free-tier T4 GPU
137
+ - Adapter file of a few MB
138
+
139
+ **Code-Switched Speech** — *Mixed language. Unified model.*
140
+ > Code-switched speech mixes languages within a sentence — e.g., "kal meeting ke baad report bhej dena." Laila handles Hindi grammar with English nouns using separate test sets and a fixed transcript convention.
141
+ - Separate Hindi, English, and code-switched test sets
142
+ - Devanagari + Latin script convention
143
+ - Noise-robust training
144
+
145
+ ---
146
+
147
+ ## 📂 Case Studies & Resources
148
+
149
+ ### Highlighted Case Studies
150
+
151
+ | Case Study | Category | Description |
152
+ |------------|----------|-------------|
153
+ | **Hindi ASR with LoRA** | ASR | Adapting Whisper-Small for Hindi speech on a single free T4 GPU |
154
+ | **Hinglish Code-Switching** | Code-Switching | Building a transcription pipeline for Hindi-English mixed speech |
155
+ | **Noise-Robust ASR** | Robustness | Training with background noise for real-world conditions |
156
+
157
+ ### Learning Resources
158
+
159
+ | Type | Title |
160
+ |------|-------|
161
+ | Paper | Laila: Parameter-Efficient Adaptation of Whisper-Small |
162
+ | Guide | Guide to LoRA Fine-Tuning |
163
+ | Guide | Guide: Whisper Architecture |
164
+ | Guide | Guide: WER & CER Evaluation |
165
+
166
+ ---
167
+
168
+ ## 📊 Benchmark Results
169
+
170
+ ### Word Error Rate (WER, %)
171
+
172
+ | Model | Hindi | Indian English | Code-Switched | Trainable Params |
173
+ |-------|-------|----------------|---------------|------------------|
174
+ | Whisper-Small (zero-shot) | TBD | TBD | TBD | 0 |
175
+ | Whisper-Small (full fine-tune) | TBD | TBD | TBD | 244M |
176
+ | **Laila (LoRA, r=32)** | **TBD** | **TBD** | **TBD** | **3.5M** |
177
+
178
+ ### Character Error Rate (CER, %)
179
+
180
+ | Model | Hindi | Indian English | Code-Switched |
181
+ |-------|-------|----------------|---------------|
182
+ | Whisper-Small (zero-shot) | TBD | TBD | TBD |
183
+ | Whisper-Small (full fine-tune) | TBD | TBD | TBD |
184
+ | **Laila (LoRA, r=32)** | **TBD** | **TBD** | **TBD** |
185
+
186
+ ### Ablation: LoRA Rank
187
+
188
+ | Rank | Trainable Params | WER (%) |
189
+ |------|------------------|---------|
190
+ | 8 | TBD | TBD |
191
+ | 16 | TBD | TBD |
192
+ | **32** | **3.5M** | **TBD** |
193
+ | 64 | TBD | TBD |
194
+
195
+ ### Ablation: Adapted Modules & Noise Augmentation
196
+
197
+ | Adapted Modules | Noise | Hindi | English | Code-Switched |
198
+ |-----------------|-------|-------|---------|---------------|
199
+ | `q_proj`, `v_proj` | Off | TBD | TBD | TBD |
200
+ | `q_proj`, `v_proj` | On | TBD | TBD | TBD |
201
+ | `q_proj`, `k_proj`, `v_proj`, `out_proj` | On | TBD | TBD | TBD |
202
+ | Decoder only | On | TBD | TBD | TBD |
203
+
204
+ ---
205
+
206
+ ## 🎙️ Indian Language Support (22 Adapters Planned)
207
+
208
+ Laila plans to support all 22 scheduled languages of India:
209
+
210
+ | Category | Languages |
211
+ |----------|-----------|
212
+ | **Indo-Aryan** | Hindi (हिन्दी), Bengali (বাংলা), Marathi (मराठी), Gujarati (ગુજરાતી), Odia (ଓଡ଼ିଆ), Punjabi (ਪੰਜਾਬੀ), Assamese (অসমীয়া), Urdu (اردو), Konkani (कोंकणी), Nepali (नेपाली), Maithili (मैथिली), Dogri (डोगरी), Sindhi (سنڌي) |
213
+ | **Dravidian** | Tamil (தமிழ்), Telugu (తెలుగు), Kannada (ಕನ್ನಡ), Malayalam (മലയാളം) |
214
+ | **Other** | Sanskrit (संस्कृतम्), Manipuri (মণিপুরী), Bodo (बड़ो), Santali (ᱥᱟᱱᱛᱟᱲᱤ), Kashmiri (کٲشُر) |
215
+ | **Code-Switching** | Hinglish, Tanglish, Benglish, Tinglish, Kanglish, Manglish, Punglish, Marathi-English |
216
+
217
+ ---
218
+
219
+ ## 📁 Dataset v1.0 Statistics
220
+
221
+ | Split | Modality | Samples | Annotated Labels |
222
+ |-------|----------|---------|------------------|
223
+ | Train | Hindi (Common Voice + AI4Bharat) | TBD | TBD |
224
+ | Val | Hindi | TBD | TBD |
225
+ | Train | Indian-Accented English | TBD | TBD |
226
+ | Val | Indian-Accented English | TBD | TBD |
227
+ | Train | Code-Switched Hindi-English | TBD | TBD |
228
+ | Val | Code-Switched Hindi-English | TBD | TBD |
229
+ | **Total** | **All** | **TBD** | **TBD** |
230
+
231
+ ---
232
+
233
+ ## 🏗️ Technology Stack
234
+
235
+ | Layer | Technologies |
236
+ |-------|--------------|
237
+ | Base Model | Whisper-Small (244M parameters) |
238
+ | Adaptation | LoRA (PEFT), rank r=32, α=64, dropout=0.05 |
239
+ | Deep Learning | PyTorch 2.0+, HuggingFace Transformers |
240
+ | Audio Processing | torchaudio, librosa, soundfile |
241
+ | Evaluation | jiwer (WER/CER), HuggingFace Evaluate |
242
+ | Training | HuggingFace Trainer, FP16, gradient checkpointing |
243
+ | Inference | PEFT + Transformers, serverless worker architecture |
244
+ | Infrastructure | 1× free-tier T4 GPU (16GB) |
245
+
246
+ ---
247
+
248
+ ## 🔮 What's Next?
249
+
250
+ - Support for all 22 Indian languages
251
+ - Real-time speech data processing
252
+ - API for companies to use
253
+ - Expansion to other code-switched pairs
254
+ - Deeper integration with AI4Bharat and Bhashini
255
+ - Expansion of noise-robust ASR for defense applications
256
+
257
+ ---
258
+
259
+ ## 📜 License
260
+
261
+ MIT — Free to use, modify, and distribute.
262
+
263
+ ---
264
+
265
+ ## 🤝 Contributing
266
+
267
+ PRs welcome! Let's build speech AI for Bharat 🇮🇳