Update the model card
Browse files
README.md
CHANGED
|
@@ -17,7 +17,13 @@ tags:
|
|
| 17 |
|
| 18 |
# Canopy-M
|
| 19 |
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
| | Urdu `urd` | Sindhi `snd` | Macro |
|
| 23 |
|---|---:|---:|---:|
|
|
@@ -30,61 +36,59 @@ Intervals are 95% speaker-cluster bootstrap. Lower is better. Run `canopy_exp_00
|
|
| 30 |
|
| 31 |

|
| 32 |
|
| 33 |
-
## Model
|
| 34 |
-
|
| 35 |
-
| | |
|
| 36 |
-
|---|---|
|
| 37 |
-
| Architecture | Conformer-CTC, Metro-style Macaron blocks |
|
| 38 |
-
| Size | 57.9M inference parameters, d=384, 12 layers, 6 heads |
|
| 39 |
-
| Languages | Urdu (id 1), Sindhi (id 2). Id 0 is unconditioned |
|
| 40 |
-
| Conditioning | language embedding and a language-ID head |
|
| 41 |
-
| Tokenizer | character, `char_urd_snd` |
|
| 42 |
-
| Frontend | 16 kHz, 80-bin log-mel, 4× conv subsampling, 25 frames/s |
|
| 43 |
-
| Objective | `(1 − 0.3) · CTC + 0.3 · auxiliary CTC` at layer 6 |
|
| 44 |
-
| Decoder in this table | greedy CTC, no n-gram |
|
| 45 |
-
| Training | 40,000 steps, bf16, RTX 3090, about 9,114 hours of audio seen, ~50/50 Urdu/Sindhi |
|
| 46 |
-
| Code | Apache-2.0 |
|
| 47 |
-
|
| 48 |
-
One block is half SwiGLU, rotary attention, a squeeze-excite depthwise convolution (kernel 31), and a second half SwiGLU, then RMSNorm. The auxiliary head is dropped at export. Sindhi keeps its own letters: Urdu yeh, kaf, and heh mappings are not applied to Sindhi.
|
| 49 |
-
|
| 50 |
## Use
|
| 51 |
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
From this snapshot, without GitHub:
|
| 55 |
-
|
| 56 |
-
```bash
|
| 57 |
-
pip install torch torchaudio soundfile numpy pyyaml huggingface_hub
|
| 58 |
-
hf auth login
|
| 59 |
-
hf download ProximaAI/Canopy-M --local-dir Canopy-M
|
| 60 |
-
python Canopy-M/canopy_m.py clip.wav --model canopy-m --language urd
|
| 61 |
-
python Canopy-M/canopy_m.py clip.wav --model ProximaAI/Canopy-M --language snd --device cpu
|
| 62 |
-
```
|
| 63 |
|
| 64 |
-
|
| 65 |
|
| 66 |
```bash
|
| 67 |
pip install torch torchaudio soundfile
|
| 68 |
pip install git+https://github.com/Proxima-AI-Co/canopy.git
|
| 69 |
hf auth login
|
| 70 |
canopy-transcribe clip.wav --model canopy-m --language urd
|
| 71 |
-
canopy-transcribe clip.wav --model ProximaAI/Canopy-M --language snd
|
| 72 |
```
|
| 73 |
|
| 74 |
```python
|
| 75 |
-
from canopy
|
| 76 |
|
| 77 |
-
asr = Canopy.from_pretrained("canopy-m")
|
| 78 |
-
asr = Canopy.from_pretrained("ProximaAI/Canopy-M") # same checkpoint
|
| 79 |
print(asr.transcribe("clip.wav", language="urd"))
|
| 80 |
print(asr.transcribe("clip.wav", language="snd"))
|
| 81 |
```
|
| 82 |
|
| 83 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
|
| 85 |
## Training data
|
| 86 |
|
| 87 |
-
About **186.7 hours** used in training. YouTube Sindhi is train-only and is excluded from the scores.
|
| 88 |
|
| 89 |
| Language | Source | Train hours | Dev hours | Labels |
|
| 90 |
|---|---|---:|---:|---|
|
|
@@ -95,7 +99,7 @@ About **186.7 hours** used in training. YouTube Sindhi is train-only and is excl
|
|
| 95 |
|
| 96 |
## Benchmark
|
| 97 |
|
| 98 |
-
|
| 99 |
|
| 100 |
| System | Params | Urdu CER | Urdu WER | Sindhi CER | Sindhi WER | Macro CER |
|
| 101 |
|---|---:|---:|---:|---:|---:|---:|
|
|
@@ -104,7 +108,7 @@ Same development clips and the same `eval_norm` for every system. Whisper is for
|
|
| 104 |
| Whisper-base | 74M | 0.855 | 1.247 | 0.624 | 1.083 | 0.739 |
|
| 105 |
| whisper-small-urdu | 244M | 0.177 | 0.330 | not run | not run | — |
|
| 106 |
|
| 107 |
-
`whisper-small-urdu` is `khawajaaliarshad/whisper-small-urdu`, an Urdu-only fine-tune, so it was not run on Sindhi. A value above 1.0 means the
|
| 108 |
|
| 109 |

|
| 110 |
|
|
@@ -121,7 +125,7 @@ Same development clips and the same `eval_norm` for every system. Whisper is for
|
|
| 121 |
|
| 122 |

|
| 123 |
|
| 124 |
-
Paired speaker-cluster
|
| 125 |
|
| 126 |
| Subset | ΔCER [95% CI] | Verdict |
|
| 127 |
|---|---|---|
|
|
@@ -144,11 +148,11 @@ whisper-small-urdu was fine-tuned on Common Voice Urdu 24.0 with its own split,
|
|
| 144 |
|
| 145 |

|
| 146 |
|
| 147 |
-
Against zero-shot Whisper-base, Canopy-M is lower on every Sindhi subset. All
|
| 148 |
|
| 149 |
### Robustness
|
| 150 |
|
| 151 |
-
Median utterance CER is 0.059 on Urdu and 0.078 on Sindhi. Canopy-M produced no runaway outputs (hypothesis longer than 3× the reference) and no empty outputs. Whisper-tiny runaway rate is 10.3% on Urdu and 9.1% on Sindhi.
|
| 152 |
|
| 153 |

|
| 154 |
|
|
@@ -156,7 +160,7 @@ Median utterance CER is 0.059 on Urdu and 0.078 on Sindhi. Canopy-M produced no
|
|
| 156 |
|
| 157 |

|
| 158 |
|
| 159 |
-
Decode real-time factor on an RTX 3090 is about 0.0006 for Canopy-M, 0.0218 for Whisper-tiny, 0.0100 for Whisper-base, and 0.0069 for whisper-small-urdu. Batch sizes differed, so treat
|
| 160 |
|
| 161 |
## Intended use
|
| 162 |
|
|
@@ -180,8 +184,9 @@ Surveillance, speaker identification, and any decision about a person made witho
|
|
| 180 |
```bibtex
|
| 181 |
@software{canopy_m_2026,
|
| 182 |
title = {Canopy-M: Conformer-CTC speech recognition for Urdu and Sindhi},
|
|
|
|
| 183 |
year = {2026},
|
| 184 |
-
url = {https://
|
| 185 |
license = {Apache-2.0}
|
| 186 |
}
|
| 187 |
```
|
|
|
|
| 17 |
|
| 18 |
# Canopy-M
|
| 19 |
|
| 20 |
+
<div align="center">
|
| 21 |
+
<img src="figures/canopy-m-logo.png" alt="Canopy-M Logo" width="220"/>
|
| 22 |
+
</div>
|
| 23 |
+
|
| 24 |
+
Canopy-M is a 57.9-million-parameter Conformer-CTC model for **Urdu** and **Sindhi**. It is non-autoregressive: a Macaron encoder predicts a token at every frame, and greedy CTC reads that sequence. This card reports the full development evaluation, with the language id on and no language model. The held-out test split has not been scored.
|
| 25 |
+
|
| 26 |
+
Code is Apache-2.0 and lives at [Proxima-AI-Co/canopy](https://github.com/Proxima-AI-Co/canopy). The weights are this repository.
|
| 27 |
|
| 28 |
| | Urdu `urd` | Sindhi `snd` | Macro |
|
| 29 |
|---|---:|---:|---:|
|
|
|
|
| 36 |
|
| 37 |

|
| 38 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
## Use
|
| 40 |
|
| 41 |
+
`canopy-m` and `ProximaAI/Canopy-M` load the same checkpoint. A later variant is the same call with another short name or Hub id. Log in with a token that can read this repository.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
|
| 43 |
+
Install the library once, then point it at any published variant:
|
| 44 |
|
| 45 |
```bash
|
| 46 |
pip install torch torchaudio soundfile
|
| 47 |
pip install git+https://github.com/Proxima-AI-Co/canopy.git
|
| 48 |
hf auth login
|
| 49 |
canopy-transcribe clip.wav --model canopy-m --language urd
|
| 50 |
+
canopy-transcribe clip.wav --model ProximaAI/Canopy-M --language snd --device cpu
|
| 51 |
```
|
| 52 |
|
| 53 |
```python
|
| 54 |
+
from canopy import Canopy
|
| 55 |
|
| 56 |
+
asr = Canopy.from_pretrained("canopy-m")
|
|
|
|
| 57 |
print(asr.transcribe("clip.wav", language="urd"))
|
| 58 |
print(asr.transcribe("clip.wav", language="snd"))
|
| 59 |
```
|
| 60 |
|
| 61 |
+
The snapshot in this repository also runs on its own:
|
| 62 |
+
|
| 63 |
+
```bash
|
| 64 |
+
pip install torch torchaudio soundfile numpy pyyaml huggingface_hub
|
| 65 |
+
hf auth login
|
| 66 |
+
hf download ProximaAI/Canopy-M --local-dir Canopy-M
|
| 67 |
+
python Canopy-M/canopy_m.py clip.wav --model canopy-m --language urd
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
`language` accepts `urd`, `ur`, `urdu`, `snd`, `sd`, and `sindhi`. Decoding in the table above is greedy CTC. Audio at any sample rate is resampled to 16 kHz mono. Files longer than 30 seconds are transcribed in chunks. If a repository ships more than one `.pt` file, pass `filename=`.
|
| 71 |
+
|
| 72 |
+
## Model
|
| 73 |
+
|
| 74 |
+
| | |
|
| 75 |
+
|---|---|
|
| 76 |
+
| Architecture | Conformer-CTC, Metro-style Macaron blocks |
|
| 77 |
+
| Size | 57.9M inference parameters, d=384, 12 layers, 6 heads |
|
| 78 |
+
| Languages | Urdu (id 1), Sindhi (id 2). Id 0 is unconditioned |
|
| 79 |
+
| Conditioning | Language embedding and a language-ID head |
|
| 80 |
+
| Tokenizer | Character, `char_urd_snd` |
|
| 81 |
+
| Frontend | 16 kHz, 80-bin log-mel, 4× conv subsampling, 25 frames/s |
|
| 82 |
+
| Objective | `(1 − 0.3) · CTC + 0.3 · auxiliary CTC` at layer 6 |
|
| 83 |
+
| Decoder in this table | Greedy CTC, no n-gram |
|
| 84 |
+
| Training | 40,000 steps, bf16, RTX 3090, about 9,114 hours of audio seen, ~50/50 Urdu/Sindhi |
|
| 85 |
+
| Licence | Apache-2.0 |
|
| 86 |
+
|
| 87 |
+
Each block is half SwiGLU, rotary attention, a squeeze-excite depthwise convolution (kernel 31), and a second half SwiGLU, closed by RMSNorm. The auxiliary head is dropped at export. Sindhi keeps its own letters. Urdu yeh, kaf, and heh mappings are not applied to Sindhi.
|
| 88 |
|
| 89 |
## Training data
|
| 90 |
|
| 91 |
+
About **186.7 hours** were used in training. YouTube Sindhi is train-only and is excluded from the scores.
|
| 92 |
|
| 93 |
| Language | Source | Train hours | Dev hours | Labels |
|
| 94 |
|---|---|---:|---:|---|
|
|
|
|
| 99 |
|
| 100 |
## Benchmark
|
| 101 |
|
| 102 |
+
Every system was scored on the same development clips with the same `eval_norm`. Whisper is forced to the `urdu` or `sindhi` token, fp16, greedy, `max_new_tokens=225`.
|
| 103 |
|
| 104 |
| System | Params | Urdu CER | Urdu WER | Sindhi CER | Sindhi WER | Macro CER |
|
| 105 |
|---|---:|---:|---:|---:|---:|---:|
|
|
|
|
| 108 |
| Whisper-base | 74M | 0.855 | 1.247 | 0.624 | 1.083 | 0.739 |
|
| 109 |
| whisper-small-urdu | 244M | 0.177 | 0.330 | not run | not run | — |
|
| 110 |
|
| 111 |
+
`whisper-small-urdu` is [`khawajaaliarshad/whisper-small-urdu`](https://huggingface.co/khawajaaliarshad/whisper-small-urdu), an Urdu-only fine-tune, so it was not run on Sindhi. A value above 1.0 means the hypothesis is longer than the reference, usually a repetition or the wrong script.
|
| 112 |
|
| 113 |

|
| 114 |
|
|
|
|
| 125 |
|
| 126 |

|
| 127 |
|
| 128 |
+
Paired speaker-cluster comparison. Δ is whisper-small-urdu minus Canopy-M. A positive value means Canopy-M makes fewer errors. “No clear difference” means the 95% interval includes zero.
|
| 129 |
|
| 130 |
| Subset | ΔCER [95% CI] | Verdict |
|
| 131 |
|---|---|---|
|
|
|
|
| 148 |
|
| 149 |

|
| 150 |
|
| 151 |
+
Against zero-shot Whisper-base, Canopy-M is lower on every Sindhi subset. All-Sindhi ΔCER is +0.517 [+0.470, +0.568]. Whisper-base did not train on this Sindhi data. Canopy-M did.
|
| 152 |
|
| 153 |
### Robustness
|
| 154 |
|
| 155 |
+
Median utterance CER is 0.059 on Urdu and 0.078 on Sindhi. Canopy-M produced no runaway outputs (a hypothesis longer than 3× the reference) and no empty outputs. Whisper-tiny’s runaway rate is 10.3% on Urdu and 9.1% on Sindhi.
|
| 156 |
|
| 157 |

|
| 158 |
|
|
|
|
| 160 |
|
| 161 |

|
| 162 |
|
| 163 |
+
Decode real-time factor on an RTX 3090 is about 0.0006 for Canopy-M, 0.0218 for Whisper-tiny, 0.0100 for Whisper-base, and 0.0069 for whisper-small-urdu. Batch sizes differed, so treat those figures as indicative.
|
| 164 |
|
| 165 |
## Intended use
|
| 166 |
|
|
|
|
| 184 |
```bibtex
|
| 185 |
@software{canopy_m_2026,
|
| 186 |
title = {Canopy-M: Conformer-CTC speech recognition for Urdu and Sindhi},
|
| 187 |
+
author = {{Proxima AI}},
|
| 188 |
year = {2026},
|
| 189 |
+
url = {https://huggingface.co/ProximaAI/Canopy-M},
|
| 190 |
license = {Apache-2.0}
|
| 191 |
}
|
| 192 |
```
|