Create README.md

README.md

@@ -0,0 +1,163 @@
+---
+language:
+  - ur
+  - sd
+license: apache-2.0
+pretty_name: Canopy-M
+pipeline_tag: automatic-speech-recognition
+library_name: pytorch
+tags:
+  - automatic-speech-recognition
+  - speech
+  - ctc
+  - conformer
+  - urdu
+  - sindhi
+---
+
+# Canopy-M
+
+Canopy-M was trained on Urdu and Sindhi. The numbers below are the evaluation.
+
+| | Urdu `urd` | Sindhi `snd` | Macro |
+|---|---:|---:|---:|
+| CER | 0.132 [0.065, 0.211] | 0.107 [0.091, 0.123] | 0.119 |
+| WER | 0.267 [0.166, 0.383] | 0.271 [0.240, 0.304] | 0.269 |
+| Utterances | 14,681 | 496 | |
+| Speakers | 63 | 164 | |
+
+
+![Canopy-M versus Whisper on the Urdu and Sindhi development sets](figures/overview_cer_wer.png)
+
+## Model
+
+| | |
+|---|---|
+| Architecture | Conformer-CTC, Metro-style Macaron blocks |
+| Size | 57.9M inference parameters, d=384, 12 layers, 6 heads |
+| Languages | Urdu (id 1), Sindhi (id 2). Id 0 is unconditioned |
+| Conditioning | language embedding and a language-ID head |
+| Tokenizer | character, `char_urd_snd` |
+| Frontend | 16 kHz, 80-bin log-mel, 4× conv subsampling, 25 frames/s |
+| Objective | `(1 − 0.3) · CTC + 0.3 · auxiliary CTC` at layer 6 |
+| Decoder in this table | greedy CTC, no n-gram |
+| Training | 40,000 steps, bf16, RTX 3090, about 9,114 hours of audio seen, ~50/50 Urdu/Sindhi |
+| Code | Apache-2.0 |
+
+
+## Use
+
+```python
+from canopy.inference.transcriber import Transcriber
+
+asr = Transcriber.from_checkpoint("canopy-m.pt", device="cuda")
+print(asr.transcribe("urdu.wav", language="urd"))
+print(asr.transcribe("sindhi.wav", language="snd"))
+```
+
+Pass `urd` or `snd`. That selects the language embedding and restricts the CTC vocabulary to that language.
+
+## Training data
+
+About **186.7 hours** used in training.
+
+| Language | Source | Train hours | Dev hours | Labels |
+|---|---|---:|---:|---|
+| Urdu | Common Voice + mahwiz + UrduSpeech | ~79.6 | ~17 | human |
+| Sindhi | Common Voice | 71.5 | 0.10 | human; mostly unvalidated `other` |
+| Sindhi | FLEURS `sd_in` | 12.3 | 1.33 | human, CC-BY-4.0 |
+
+## Benchmark
+
+Same development clips and the same `eval_norm` for every system. Whisper is forced to the `urdu` or `sindhi` token, fp16, greedy, `max_new_tokens=225`.
+
+| System | Params | Urdu CER | Urdu WER | Sindhi CER | Sindhi WER | Macro CER |
+|---|---:|---:|---:|---:|---:|---:|
+| **Canopy-M** | 57.9M | **0.132** | **0.267** | **0.107** | **0.271** | **0.119** |
+| Whisper-tiny | 39M | 1.024 | 1.477 | 1.430 | 1.599 | 1.227 |
+| Whisper-base | 74M | 0.855 | 1.247 | 0.624 | 1.083 | 0.739 |
+| whisper-small-urdu | 244M | 0.177 | 0.330 | not run | not run | — |
+
+`whisper-small-urdu` is `khawajaaliarshad/whisper-small-urdu`, an Urdu-only fine-tune, so it was not run on Sindhi. A value above 1.0 means the output is longer than the reference, usually a repetition or the wrong script.
+
+![Model size against median utterance CER](figures/size_vs_cer.png)
+
+### Urdu by source
+
+14,681 utterances, 63 speakers: Common Voice 7,772 / 54 speakers, UrduSpeech 4,296 / 7, mahwiz 2,613 / 2.
+
+| System | CV CER | UrduSpeech CER | mahwiz CER |
+|---|---:|---:|---:|
+| Canopy-M | 0.248 | 0.022 | 0.027 |
+| Whisper-tiny | 1.505 | 0.642 | 0.451 |
+| Whisper-base | 1.520 | 0.209 | 0.283 |
+| whisper-small-urdu | 0.278 | 0.069 | 0.111 |
+
+![Urdu development CER and WER by source](figures/urdu_by_source.png)
+
+Paired speaker-cluster test, Δ = whisper-small-urdu minus Canopy-M. Positive means Canopy-M makes fewer errors. “No clear difference” means the 95% interval includes zero.
+
+| Subset | ΔCER [95% CI] | Verdict |
+|---|---|---|
+| All Urdu | +0.046 [−0.003, +0.118] | no clear difference |
+| Common Voice | +0.030 [−0.042, +0.184] | no clear difference |
+| UrduSpeech | +0.047 [+0.021, +0.090] | Canopy-M lower CER |
+| mahwiz | +0.084 [+0.074, +0.099] | Canopy-M lower CER |
+
+whisper-small-urdu was fine-tuned on Common Voice Urdu 24.0 with its own split, so its Common Voice number may include clips it trained on. Canopy-M trained on the UrduSpeech and mahwiz training splits, so those two rows are in-domain for Canopy-M.
+
+### Sindhi by source
+
+496 utterances, 164 speakers: Common Voice 70 / 14 speakers, FLEURS 426 / 150 speakers. FLEURS `sd_in` is India-recorded Sindhi.
+
+| System | CV CER | FLEURS CER |
+|---|---:|---:|
+| Canopy-M | 0.286 | 0.095 |
+| Whisper-tiny | 1.911 | 1.399 |
+| Whisper-base | 0.690 | 0.619 |
+
+![Sindhi development CER and WER by source](figures/sindhi_by_source.png)
+
+Against zero-shot Whisper-base, Canopy-M is lower on every Sindhi subset. All Sindhi ΔCER is +0.517 [+0.470, +0.568]. Whisper-base did not train on this Sindhi data. Canopy-M did.
+
+### Robustness
+
+Median utterance CER is 0.059 on Urdu and 0.078 on Sindhi. Canopy-M produced no runaway outputs (hypothesis longer than 3× the reference) and no empty outputs. Whisper-tiny runaway rate is 10.3% on Urdu and 9.1% on Sindhi.
+
+![Per-utterance CER](figures/utterance_cer_distribution.png)
+
+![Runaway rate](figures/runaway_rate.png)
+
+![Paired differences](figures/paired_differences.png)
+
+Decode real-time factor on an RTX 3090 is about 0.0006 for Canopy-M, 0.0218 for Whisper-tiny, 0.0100 for Whisper-base, and 0.0069 for whisper-small-urdu. Batch sizes differed, so treat the speed column as indicative.
+
+## Intended use
+
+Research transcription of read Urdu and Sindhi at 16 kHz, one language per utterance, with the language given.
+
+## Out of scope
+
+Surveillance, speaker identification, and any decision about a person made without review. Production deployment while the test split is unscored. Telephone, broadcast, conversational, and code-switched speech.
+
+## Limitations
+
+- These scores are the development split. The test split (Urdu ~37 h, Sindhi FLEURS ~3.3 h plus a small Common Voice test) has not been opened.
+- Sindhi Common Voice development audio is 0.10 hours (70 utterances). Most of the Sindhi score is FLEURS.
+- Common Voice Sindhi training audio is mostly unvalidated `other`, included with `--include-unvalidated`.
+- Urdu Common Voice CER is 0.248. The headline 0.132 is pulled down by UrduSpeech and mahwiz.
+- A wrong language id can cross Urdu and Sindhi, which share a Perso-Arabic script.
+- No word n-gram was used in this table. Fusion can still overwrite a correct acoustic hypothesis.
+
+## Citation
+
+```bibtex
+@software{canopy_m_2026,
+  title = {Canopy-M: Conformer-CTC speech recognition for Urdu and Sindhi},
+  year = {2026},
+  url = {https://github.com/mwzkhalil/canopy},
+  license = {Apache-2.0}
+}
+```
+
+Also cite Common Voice, FLEURS, and the Metro-ASR / Conformer designs the block follows.