| --- |
| language: ur |
| license: apache-2.0 |
| base_model: openai/whisper-large-v3-turbo |
| library_name: peft |
| tags: |
| - whisper |
| - urdu |
| - speech-recognition |
| - telephone-audio |
| - lora |
| --- |
| |
| # PakUrdu-Conversational-ASR — Whisper Large-v3 Turbo Urdu Call Fine-tune (LoRA) |
|
|
| LoRA adapter (r=16, α=32, attention projections) on top of |
| `openai/whisper-large-v3-turbo` (immutable revision |
| `41f01f3fe87f28c78e2fbf8b568835947dd65ed9`), fine-tuned for |
| **telephone-domain conversational Urdu** with a two-stage curriculum — broad |
| public-domain Urdu speech first, then specialization on real customer-call |
| audio from **SimpleRishta** (سمپل رشتہ), PII-masked, speaker-safe splits. No |
| SimpleRishta audio or transcripts are published; only these adapter weights. |
| This release (r14) supersedes r12 (8.9%) on the same frozen evaluation. |
|
|
| **Live demo:** [PakUrdu-ASR-Demo](https://huggingface.co/spaces/KhiredNetworks/PakUrdu-ASR-Demo) |
| — runs fully in your browser (WebGPU); your audio never leaves your device. |
|
|
| **Full quality, no GPU of your own:** open [`inference_notebook.ipynb`](inference_notebook.ipynb) |
| in [Google Colab](https://colab.research.google.com/#fileId=https%3A%2F%2Fhuggingface.co%2FKhiredNetworks%2FPakUrdu-Conversational-ASR%2Fresolve%2Fmain%2Finference_notebook.ipynb) |
| or Kaggle — a free cloud GPU runs the complete unquantized model on your clip. |
|
|
| > ✅ **Verified end-to-end** — the 7.96% headline number is reproduced live in |
| > a public **[Kaggle proof notebook: e1-sr-proof-r14-sr133](https://www.kaggle.com/code/adeelejaz8611/e1-sr-proof-r14-sr133)** |
| > (full details under Results). |
|
|
| ## Intended / out-of-scope use |
|
|
| - **Intended:** transcribing Urdu telephone-call audio (customer-call |
| traffic, conversational telephony speech), decoded with `language=ur`, |
| `task=transcribe`, greedy. |
| - **Out of scope:** general-purpose studio/broadcast Urdu (our internal |
| measurements place it behind general-purpose fine-tunes there), languages |
| other than Urdu, speaker identification, or any decision-making about |
| individuals. |
|
|
| ## Results |
|
|
| 133 held-out SimpleRishta calls, zero caller overlap with training, corpus |
| C0 WER (normalized, PII-masked), identical decode settings for both models: |
|
|
| | Model | WER on SR calls | |
| |---|---| |
| | `openai/whisper-large-v3` (off-the-shelf) | 15.58% | |
| | **this adapter** | **7.96%** | |
|
|
| Reference transcription: an independent high-accuracy Whisper-family model |
| (eval-only, never trained on). Both models scored identically against it. |
| Head-to-head on the same calls vs our previous in-house baseline (17.3%): |
| this adapter wins 120 / loses 9 / ties 4. |
| Zero degenerate generations across all stability checks (repetition-rate and |
| length-ratio guards, clean greedy decoding). Aggregate protocol and numbers: |
| [`eval/aggregate-results.json`](eval/aggregate-results.json). |
|
|
| **Independent reproduction (live run):** the 7.96% figure is re-derived |
| end-to-end on a clean GPU in this public Kaggle notebook — model loaded, |
| 133 calls decoded, fresh WER printed against the certified value (observed |
| drift 0.0000): |
| [Kaggle proof notebook: e1-sr-proof-r14-sr133](https://www.kaggle.com/code/adeelejaz8611/e1-sr-proof-r14-sr133). |
|
|
| ## Usage |
|
|
| ```python |
| import torch, soundfile as sf |
| from peft import PeftModel |
| from transformers import WhisperForConditionalGeneration, WhisperProcessor |
| |
| base = "openai/whisper-large-v3-turbo" |
| processor = WhisperProcessor.from_pretrained(base) |
| model = WhisperForConditionalGeneration.from_pretrained(base) |
| model = PeftModel.from_pretrained(model, "KhiredNetworks/PakUrdu-Conversational-ASR").merge_and_unload() |
| |
| speech, sr = sf.read("call.wav", dtype="float32") |
| inputs = processor(speech, sampling_rate=sr, return_tensors="pt") |
| ids = model.generate(inputs.input_features, language="ur", |
| task="transcribe", do_sample=False) |
| print(processor.batch_decode(ids, skip_special_tokens=True)[0]) |
| ``` |
|
|
| A runnable version ships as [`inference_example.py`](inference_example.py). |
| To run the complete model on a free cloud GPU — no GPU of your own, nothing |
| installed — open [`inference_notebook.ipynb`](inference_notebook.ipynb) |
| (works in Google Colab and Kaggle). |
|
|
| ## Training data & governance |
|
|
| - Fine-tuned on real SimpleRishta customer-call audio (16 kHz telephone |
| Urdu; transcripts PII-masked before training; brand name preserved; |
| caller-group-safe train/dev split — no caller appears in both). |
| - Evaluation set frozen and never used in training. |
| - Training method: LoRA (PEFT), two-stage curriculum (stage 1: broad |
| public-domain Urdu speech, one epoch; stage 2: telephony specialization |
| with public-data replay, one epoch), fresh from the pinned base revision — |
| no continuation from any other checkpoint. |
| - No SimpleRishta audio, transcripts, or caller-identifying data are |
| published. Published results are aggregate-only. |
| - Full training/eval audit trail retained internally (sealed compute |
| records; certified numbers independently reproduced end-to-end on a |
| clean GPU run, drift 0.0000). |
|
|
| ## Transcription convention & inference settings |
|
|
| Urdu script output; evaluation text normalized under a corpus C0 protocol |
| (Unicode normalization, digit/punctuation folding, brand canonicalization, |
| PII masking) before scoring. Decode: greedy (`do_sample=False`), |
| `language=ur`, `task=transcribe`, 16 kHz mono input. FP16 on GPU / FP32 on |
| CPU both supported; results certified on a single T4. |
|
|
| ## Limitations |
|
|
| - Scores are vs a model reference, not human transcription. |
| - Evaluated on accepted (clean) call audio; performance on heavily noisy |
| or overlapped speech is not characterized here. |
| - Urdu only (`language=ur` fixed in the intended decode config). |
|
|
| ## License, attribution & contact |
|
|
| - Code/adapter: Apache-2.0 (see `LICENSE`). Base model: |
| `openai/whisper-large-v3-turbo` — Whisper license and attribution chain |
| apply downstream. |
| - Citation: see `CITATION.cff` ([Khired Networks](https://huggingface.co/KhiredNetworks)). |
| - Issues: use the repo's Issues tab. |
|
|
