| <div align="center"> |
|
|
| <!-- Laila Logo --> |
| <img src="https://github.com/user-attachments/assets/560b6d36-7aef-4d72-bcac-1171fcb9e8c9" alt="Laila Logo" width="300"/> |
|
|
| ### Parameter-Efficient Adaptation of Whisper-Small for Indian-Accented and Code-Switched Hindi-English Speech |
|
|
| [](https://github.com/techindro) |
| [](https://opensource.org/licenses/MIT) |
| [](https://www.python.org/downloads/) |
| [](https://pytorch.org/) |
| [](https://huggingface.co/docs/transformers) |
| [](https://github.com/huggingface/peft) |
| [](https://neurips.cc) |
|
|
| **3.5M trainable params · 1.5% of base model · Trains on 1 free T4 GPU** |
|
|
| </div> |
|
|
| --- |
|
|
| ## 🌐 Platform Overview |
|
|
| > *"Whisper, but for India — parameter-efficient ASR for Indian-accented and code-switched speech"* |
|
|
| Laila provides high-quality speech recognition, model evaluation, and AI tools designed for Indian users, enterprises, and researchers working on next-generation voice applications. |
|
|
| --- |
|
|
| ## 🚀 Live Demos |
|
|
| Experience Laila in action. All demos run on Hugging Face Spaces — no installation required. |
|
|
| | Demo | Description | Link | |
| |------|-------------|------| |
| | 🎙️ Hindi ASR | Hindi speech recognition with LoRA-tuned Whisper | [Try Now](https://huggingface.co/spaces/techindro/Laila-Demo) | |
| | 🔀 Hinglish ASR | Code-switched Hindi-English transcription | [Try Now](https://huggingface.co/spaces/techindro/Laila-Hinglish) | |
| | 🇮🇳 Indian English ASR | Indian-accented English transcription | [Try Now](https://huggingface.co/spaces/techindro/Laila-IndianEnglish) | |
| | 🌐 22-Language Suite | One adapter per scheduled Indian language | [Coming Soon](https://huggingface.co/techindro) | |
|
|
| 🌐 Repository: [github.com/techindro/Laila](https://github.com/techindro/Laila) |
|
|
| --- |
|
|
| ## 📖 What is Laila? |
|
|
| Laila is a **parameter-efficient speech recognition model** purpose-built for India's linguistic diversity. It adapts the base Whisper-Small model using LoRA, training only **1.5% of parameters** while keeping all 244M base weights frozen. |
|
|
| ### The Name |
|
|
| - **Laila** = "Night" in Arabic/Persian, "play" in Hindi — evoking speech and expression |
| - **Model** = Speech recognition for Indian languages |
|
|
| So Laila means bringing clarity to Indian speech, day or night. |
|
|
| ### What Problem Does It Solve? |
|
|
| Global speech models like Whisper, trained mostly on Western English, make more errors on Indian speech because: |
|
|
| 1. Indian-accented English has different phonetics, rhythm, and stress patterns |
| 2. Everyday speech is often code-switched (Hindi + English in one sentence) |
| 3. No single accepted transcript convention exists (Devanagari vs. Roman) |
| 4. Full fine-tuning is costly — needs more memory and produces full model copies |
|
|
| **The result:** Speech recognition that works perfectly in San Francisco but struggles in Mumbai, Delhi, and Chennai. |
|
|
| ### How Does Laila Work? |
|
|
| The pipeline has five parts that work together: |
|
|
| | Part | What It Does | |
| |------|---------------| |
| | 🎧 Audio Preprocessing | Resampling, mono conversion, normalization, 30s chunking | |
| | 🧠 Whisper-Small | Frozen 244M-parameter encoder-decoder Transformer | |
| | 🔗 LoRA Adapters | Trains 3.5M params on `q_proj` & `v_proj` attention layers | |
| | 🔊 Noise Augmentation | Mixes real background noise at random SNRs (train only) | |
| | 📝 Transcript Convention | Devanagari for Hindi, Latin for English, normalized scoring | |
|
|
| --- |
|
|
| ## 📊 Key Results at a Glance |
|
|
| | Metric | Laila | Whisper-Small (zero-shot) | Improvement | |
| |--------|-------|---------------------------|-------------| |
| | Trainable Parameters | 3.5M | 244M | **1.5% of base** | |
| | Adapter Size | ~few MB | 244M | **~70× smaller** | |
| | Hardware | 1× T4 (16GB) | 1× T4 (16GB) | **Free-tier** | |
| | Hindi WER | TBD | TBD | TBD | |
| | Indian-English WER | TBD | TBD | TBD | |
| | Code-Switched WER | TBD | TBD | TBD | |
|
|
| --- |
|
|
| ## 🎯 The Problem |
|
|
| **Global ASR training data ignores 1.4 billion Indian voices.** |
|
|
| Existing models like Whisper were trained on 680k hours of mostly English data: |
|
|
| | Limitation | Consequence | |
| |------------|-------------| |
| | Western English bias | Errors on Indian-accented English | |
| | Monolingual training | Fails on code-switched speech | |
| | Mixed-script ambiguity | Inconsistent training/evaluation | |
| | Full fine-tuning cost | High memory, full model copies per variant | |
|
|
| --- |
|
|
| ## 🚀 The Solution |
|
|
| Laila is a **parameter-efficient adaptation** of Whisper-Small purpose-built for India's linguistic diversity. |
|
|
| ### Comparison with Baselines |
|
|
| | Feature | Whisper-Small (zero-shot) | Whisper-Small (full FT) | Laila (LoRA) | |
| |---------|---------------------------|--------------------------|--------------| |
| | Hindi Support | Partial | ✅ Yes | ✅ Yes | |
| | Indian-Accented English | Partial | ✅ Yes | ✅ Yes | |
| | Code-Switched (Hinglish) | ❌ | ✅ Yes | ✅ Yes | |
| | Trainable Params | 0 | 244M | **3.5M** | |
| | Adapter Size | — | Full model | **Few MB** | |
| | Memory (Training) | — | High | **16GB T4** | |
| | Base Model Weights | Frozen | Fully trained | **Frozen** | |
|
|
| --- |
|
|
| ## 🧠 AI Knowledge Hub |
|
|
| Laila explores core pillars of modern speech AI and how LoRA applies each discipline to solve critical language challenges. |
|
|
| ### Core AI Disciplines |
|
|
| **Parameter-Efficient Fine-Tuning** — *Train less. Do more.* |
| > LoRA writes the update to a weight matrix as the product of two low-rank matrices and trains only these. This lowers memory use and the size of the stored update. Laila leverages LoRA to adapt Whisper-Small with just 1.5% of trainable parameters. |
| - 3.5M trainable params vs. 244M base |
| - Single free-tier T4 GPU |
| - Adapter file of a few MB |
|
|
| **Code-Switched Speech** — *Mixed language. Unified model.* |
| > Code-switched speech mixes languages within a sentence — e.g., "kal meeting ke baad report bhej dena." Laila handles Hindi grammar with English nouns using separate test sets and a fixed transcript convention. |
| - Separate Hindi, English, and code-switched test sets |
| - Devanagari + Latin script convention |
| - Noise-robust training |
|
|
| --- |
|
|
| ## 📂 Case Studies & Resources |
|
|
| ### Highlighted Case Studies |
|
|
| | Case Study | Category | Description | |
| |------------|----------|-------------| |
| | **Hindi ASR with LoRA** | ASR | Adapting Whisper-Small for Hindi speech on a single free T4 GPU | |
| | **Hinglish Code-Switching** | Code-Switching | Building a transcription pipeline for Hindi-English mixed speech | |
| | **Noise-Robust ASR** | Robustness | Training with background noise for real-world conditions | |
|
|
| ### Learning Resources |
|
|
| | Type | Title | |
| |------|-------| |
| | Paper | Laila: Parameter-Efficient Adaptation of Whisper-Small | |
| | Guide | Guide to LoRA Fine-Tuning | |
| | Guide | Guide: Whisper Architecture | |
| | Guide | Guide: WER & CER Evaluation | |
|
|
| --- |
|
|
| ## 📊 Benchmark Results |
|
|
| ### Word Error Rate (WER, %) |
|
|
| | Model | Hindi | Indian English | Code-Switched | Trainable Params | |
| |-------|-------|----------------|---------------|------------------| |
| | Whisper-Small (zero-shot) | TBD | TBD | TBD | 0 | |
| | Whisper-Small (full fine-tune) | TBD | TBD | TBD | 244M | |
| | **Laila (LoRA, r=32)** | **TBD** | **TBD** | **TBD** | **3.5M** | |
|
|
| ### Character Error Rate (CER, %) |
|
|
| | Model | Hindi | Indian English | Code-Switched | |
| |-------|-------|----------------|---------------| |
| | Whisper-Small (zero-shot) | TBD | TBD | TBD | |
| | Whisper-Small (full fine-tune) | TBD | TBD | TBD | |
| | **Laila (LoRA, r=32)** | **TBD** | **TBD** | **TBD** | |
|
|
| ### Ablation: LoRA Rank |
|
|
| | Rank | Trainable Params | WER (%) | |
| |------|------------------|---------| |
| | 8 | TBD | TBD | |
| | 16 | TBD | TBD | |
| | **32** | **3.5M** | **TBD** | |
| | 64 | TBD | TBD | |
|
|
| ### Ablation: Adapted Modules & Noise Augmentation |
|
|
| | Adapted Modules | Noise | Hindi | English | Code-Switched | |
| |-----------------|-------|-------|---------|---------------| |
| | `q_proj`, `v_proj` | Off | TBD | TBD | TBD | |
| | `q_proj`, `v_proj` | On | TBD | TBD | TBD | |
| | `q_proj`, `k_proj`, `v_proj`, `out_proj` | On | TBD | TBD | TBD | |
| | Decoder only | On | TBD | TBD | TBD | |
|
|
| --- |
|
|
| ## 🎙️ Indian Language Support (22 Adapters Planned) |
|
|
| Laila plans to support all 22 scheduled languages of India: |
|
|
| | Category | Languages | |
| |----------|-----------| |
| | **Indo-Aryan** | Hindi (हिन्दी), Bengali (বাংলা), Marathi (मराठी), Gujarati (ગુજરાતી), Odia (ଓଡ଼ିଆ), Punjabi (ਪੰਜਾਬੀ), Assamese (অসমীয়া), Urdu (اردو), Konkani (कोंकणी), Nepali (नेपाली), Maithili (मैथिली), Dogri (डोगरी), Sindhi (سنڌي) | |
| | **Dravidian** | Tamil (தமிழ்), Telugu (తెలుగు), Kannada (ಕನ್ನಡ), Malayalam (മലയാളം) | |
| | **Other** | Sanskrit (संस्कृतम्), Manipuri (মণিপুরী), Bodo (बड़ो), Santali (ᱥᱟᱱᱛᱟᱲᱤ), Kashmiri (کٲشُر) | |
| | **Code-Switching** | Hinglish, Tanglish, Benglish, Tinglish, Kanglish, Manglish, Punglish, Marathi-English | |
|
|
| --- |
|
|
| ## 📁 Dataset v1.0 Statistics |
|
|
| | Split | Modality | Samples | Annotated Labels | |
| |-------|----------|---------|------------------| |
| | Train | Hindi (Common Voice + AI4Bharat) | TBD | TBD | |
| | Val | Hindi | TBD | TBD | |
| | Train | Indian-Accented English | TBD | TBD | |
| | Val | Indian-Accented English | TBD | TBD | |
| | Train | Code-Switched Hindi-English | TBD | TBD | |
| | Val | Code-Switched Hindi-English | TBD | TBD | |
| | **Total** | **All** | **TBD** | **TBD** | |
|
|
| --- |
|
|
| ## 🏗️ Technology Stack |
|
|
| | Layer | Technologies | |
| |-------|--------------| |
| | Base Model | Whisper-Small (244M parameters) | |
| | Adaptation | LoRA (PEFT), rank r=32, α=64, dropout=0.05 | |
| | Deep Learning | PyTorch 2.0+, HuggingFace Transformers | |
| | Audio Processing | torchaudio, librosa, soundfile | |
| | Evaluation | jiwer (WER/CER), HuggingFace Evaluate | |
| | Training | HuggingFace Trainer, FP16, gradient checkpointing | |
| | Inference | PEFT + Transformers, serverless worker architecture | |
| | Infrastructure | 1× free-tier T4 GPU (16GB) | |
|
|
| --- |
|
|
| ## 🔮 What's Next? |
|
|
| - Support for all 22 Indian languages |
| - Real-time speech data processing |
| - API for companies to use |
| - Expansion to other code-switched pairs |
| - Deeper integration with AI4Bharat and Bhashini |
| - Expansion of noise-robust ASR for defense applications |
|
|
| --- |
|
|
| ## 📜 License |
|
|
| MIT — Free to use, modify, and distribute. |
|
|
| --- |
|
|
| ## 🤝 Contributing |
|
|
| PRs welcome! Let's build speech AI for Bharat 🇮🇳 |
