fihria commited on
Commit
032e1cc
·
verified ·
1 Parent(s): be1cdf6

Update model card instructions

Browse files
Files changed (1) hide show
  1. README.md +156 -55
README.md CHANGED
@@ -16,7 +16,7 @@ tags:
16
  - code-switching
17
  ---
18
 
19
- # fihrIA-ASR (0.6B & 1.7B)
20
 
21
  fihrIA-ASR is fihrIA lab's open-weight ASR model family purpose-built for Moroccan Darija.
22
  Based on Qwen3-ASR, trained using a shared 89k-token Darija tokenizer and evaluated under a
@@ -32,11 +32,29 @@ unified protocol, the family is built for robust, real-world transcription:
32
  - **1.7B:** engineered as the high-accuracy ceiling and setting up a new SOTA for
33
  open-weight Darija ASR models.
34
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
35
  ## Technical & deployment details
36
 
37
  This repository hosts the unquantized **BF16 safetensors** weights, configs, and 89k Darija
38
- tokenizer for use with `qwen-asr`, `transformers`, and `vLLM`. If you are deploying on CPU or
39
- with `llama.cpp`, use the matching GGUF repositories below.
 
40
 
41
  Repositories are hosted separately by size and format, sharing an identical layout:
42
 
@@ -49,14 +67,19 @@ Repositories are hosted separately by size and format, sharing an identical layo
49
 
50
  **State of the art among open-weight Darija ASR models.** Both sizes beat every open-weight system we tested on held-out meaning error: the 1.7B reaches **6.1% SER** (vs 8.4% for Mistral's Voxtral Mini 4B and 11.2% for MoulSot) and achieves the best DarijaCS CER on both test sets, while running 7.1× faster than Voxtral on one H100. The 0.6B still beats Voxtral at **7.1% SER** in a laptop-sized footprint — 7.5× faster than Voxtral on H100 and 5.98× real time at 4-bit on a laptop CPU.
51
 
 
 
52
  | | fihrIA-ASR 0.6B | fihrIA-ASR 1.7B | Mistral Voxtral Mini 4B | MoulSot 1.7B |
53
  |---|---|---|---|---|
54
  | Held-out mean SER ↓ | **7.1%** | **6.1%** | 8.4% | 11.2% |
55
- | DarijaCS CER ↓ (Prism / Probe) | **7.98 / 9.80%** | **6.24 / 8.71%** | 6.68 / 9.69% | 10.27 / 14.93% |
56
  | H100 throughput ↑ | **255×** real time | **243×** real time | 34× | 147× |
57
  | Laptop CPU, Q4 ↑ | **5.98×** | — | — | 3.06× |
58
  | Footprint, BF16 / Q4 | 1,447 / 721 MB | 3,830 / 1,642 MB | — | — |
59
 
 
 
 
60
  ![Mean semantic error rate against Mistral Voxtral Mini 4B and MoulSot](comparison-ser.png)
61
 
62
  ![Accuracy against throughput on one H100](comparison-speed-accuracy.png)
@@ -65,33 +88,38 @@ Held-out numbers are the mean of four independent LLM judges over 508 clips. Vox
65
  DarijaCS figures are its own published numbers; everything else is our rerun on the same
66
  clips. Lower is better for SER and CER, higher for throughput.
67
 
68
- ## Contact
69
-
70
- fihrIA is a Moroccan AI research lab working on Darija speech and language. Questions about
71
- the model, the data, or a collaboration: **contact@fihria.ma**.
72
-
73
  ## Files
74
 
75
- Each safetensors repository carries its own size (`0.6B` or `1.7B`) in BF16:
76
-
77
- | File | Size (0.6B / 1.7B) | Description |
78
  |---|---|---|
79
- | `model.safetensors` | 1.44 GB / 3.82 GB | Full BF16 weights (audio encoder + multimodal projector + language model). |
80
  | `config.json`, `generation_config.json`, `preprocessor_config.json` | — | Model, generation, and audio feature extractor configs. |
81
- | `tokenizer.json`, `vocab.json`, `merges.txt`, `tokenizer_config.json`, `special_tokens_map.json`, `chat_template.json` | — | Shared 89k-token Darija tokenizer (do **not** swap with the 151k Qwen base tokenizer). |
 
82
 
83
- ## Quick start
84
 
85
- Install the [`qwen-asr`](https://github.com/QwenLM/Qwen3-ASR) package (supports both the
86
- `transformers` and `vLLM` backends):
87
 
88
  ```bash
89
  pip install -U qwen-asr
90
- # or with vLLM backend:
91
  # pip install -U "qwen-asr[vllm]"
92
  ```
93
 
94
- ### Python (`qwen-asr` / transformers backend)
 
 
 
 
 
 
 
 
 
 
 
 
95
 
96
  ```python
97
  import torch
@@ -108,17 +136,68 @@ model = Qwen3ASRModel.from_pretrained(
108
 
109
  results = model.transcribe(
110
  audio="sample.wav",
111
- language=None, # or "Darija"
112
  )
113
 
114
  print(results[0].language)
115
  print(results[0].text)
116
  ```
117
 
118
- ### vLLM backend
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119
 
120
  ```python
121
- import torch
122
  from qwen_asr import Qwen3ASRModel
123
 
124
  model = Qwen3ASRModel.LLM(
@@ -132,11 +211,12 @@ results = model.transcribe(audio="sample.wav", language=None)
132
  print(results[0].text)
133
  ```
134
 
 
 
135
  ## Prompt format
136
 
137
- `qwen-asr` applies the canonical prompt automatically. If you call the model directly through
138
- another runtime, use an empty `system` turn and an **audio-only `user` turn** (no instruction
139
- text):
140
 
141
  ```
142
  <|im_start|>system\n<|im_end|>\n
@@ -144,62 +224,76 @@ text):
144
  <|im_start|>assistant\n
145
  ```
146
 
147
- ## Output format
148
-
149
- Raw decoder outputs emit the detected language followed by `<asr_text>` and the transcript
150
- (handled automatically by `qwen-asr`):
151
-
152
- <div dir="rtl" lang="ar">‏language Darija<asr_text>التلفون ديالي طاح وتهرس ليه l'écran</div>
153
-
154
- If you decode directly without `qwen-asr`, parse both forms (with or without `<asr_text>`):
155
 
156
- ```python
157
- import re
158
 
159
- ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)
 
160
 
161
- def parse(raw: str) -> tuple[str | None, str]:
162
- m = ASR_RE.search(raw.strip())
163
- return (m.group(1), m.group(2).strip()) if m else (None, raw.strip())
164
- ```
 
165
 
166
- The language name is ASCII and runs directly into Arabic text, so match on `[A-Za-z]+`
167
- rather than `\S+` — otherwise the first Arabic word gets swallowed into the language field.
168
 
169
  ## Reference output
170
 
171
- Use this to verify your setup. Reference clip: 3.11 s, 16 kHz mono WAV, DarijaCS Probe,
172
- code-switched Darija–French. The transcript below is the 0.6B; the 1.7B runs through the
173
- same pipeline with its own weights.
174
 
175
  Ground truth:
176
 
177
  <div dir="rtl" lang="ar">‏الtéléphone ديالي طاح وتهرس l'écran.</div>
178
 
179
- `BF16` transcript after stripping the `language Darija` prefix and the `<asr_text>` marker:
180
 
181
  <div dir="rtl" lang="ar">‏التلفون ديالي طاح وتهرس ليه l'écran</div>
182
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
183
  The model writes the French word in Arabic script (`الtéléphone` → `التلفون`) and adds `ليه`
184
- before `l'écran`.
185
 
186
  ## Checksums (MD5)
187
 
188
- 0.6B (`fihriaLab/fihrIA-ASR-0.6B`):
189
 
190
  ```
191
- 0c49e982df1c01adda76b886643f14e8 model.safetensors
192
  ```
193
 
194
- 1.7B (`fihriaLab/fihrIA-ASR-1.7B`):
195
 
196
  ```
197
- e9e03c692d75413fbc94479e73d8c8ae model.safetensors
198
  ```
199
 
200
- ## Citation
201
 
202
- If you use fihrIA-ASR, please cite the paper:
 
 
 
 
 
203
 
204
  ```bibtex
205
  @misc{fihria2026asr,
@@ -223,5 +317,12 @@ If you use fihrIA-ASR, please cite the paper:
223
  - Vocal music is transcribed as lyrics, and very noisy or instrumental clips can occasionally
224
  produce short text. For noisy production pipelines, gating with a lightweight VAD (e.g.,
225
  Silero VAD) is recommended.
226
- - `transformers` support for the `qwen3_asr` architecture is recent; use `qwen-asr` or a
227
- recent `transformers` release, or use the GGUF builds with `llama.cpp`.
 
 
 
 
 
 
 
 
16
  - code-switching
17
  ---
18
 
19
+ # fihrIA-ASR-1.7B
20
 
21
  fihrIA-ASR is fihrIA lab's open-weight ASR model family purpose-built for Moroccan Darija.
22
  Based on Qwen3-ASR, trained using a shared 89k-token Darija tokenizer and evaluated under a
 
32
  - **1.7B:** engineered as the high-accuracy ceiling and setting up a new SOTA for
33
  open-weight Darija ASR models.
34
 
35
+ This repository holds the unquantized **BF16** weights of the 1.7B, the accuracy ceiling of the
36
+ family. It is the GPU path; for CPU deployment use the
37
+ [GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF).
38
+
39
+ ## Which build do I need?
40
+
41
+ | Deployment | Build | Notes |
42
+ |---|---|---|
43
+ | CPU (laptop, edge, no GPU) | [GGUF Q4_K_M](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF) | see the GGUF card |
44
+ | NVIDIA GPU | **this repo** (BF16) | `qwen-asr`, transformers backend |
45
+ | NVIDIA GPU, high throughput | **this repo** + vLLM | batching, `qwen-asr[vllm]` |
46
+ | Edge / real-time laptop CPU | [0.6B](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B) | smaller, faster, slightly higher SER |
47
+
48
+ **Memory:** 1.7B BF16 needs roughly 8 GB of VRAM (3.82 GB of weights plus KV cache and
49
+ activations). The 0.6B needs about 4 GB. BF16 inference on CPU runs far below real time — use
50
+ GGUF there.
51
+
52
  ## Technical & deployment details
53
 
54
  This repository hosts the unquantized **BF16 safetensors** weights, configs, and 89k Darija
55
+ tokenizer. Inference runs through [`qwen-asr`](https://github.com/QwenLM/Qwen3-ASR), which
56
+ provides both the `transformers` and `vLLM` backends and registers the `qwen3_asr` architecture
57
+ with `transformers` on import.
58
 
59
  Repositories are hosted separately by size and format, sharing an identical layout:
60
 
 
67
 
68
  **State of the art among open-weight Darija ASR models.** Both sizes beat every open-weight system we tested on held-out meaning error: the 1.7B reaches **6.1% SER** (vs 8.4% for Mistral's Voxtral Mini 4B and 11.2% for MoulSot) and achieves the best DarijaCS CER on both test sets, while running 7.1× faster than Voxtral on one H100. The 0.6B still beats Voxtral at **7.1% SER** in a laptop-sized footprint — 7.5× faster than Voxtral on H100 and 5.98× real time at 4-bit on a laptop CPU.
69
 
70
+ BF16 weights (this repo):
71
+
72
  | | fihrIA-ASR 0.6B | fihrIA-ASR 1.7B | Mistral Voxtral Mini 4B | MoulSot 1.7B |
73
  |---|---|---|---|---|
74
  | Held-out mean SER ↓ | **7.1%** | **6.1%** | 8.4% | 11.2% |
75
+ | DarijaCS CER ↓ (Prism / Probe, BF16) | **7.98 / 9.80%** | **6.24 / 8.71%** | 6.68 / 9.69% | 10.27 / 14.93% |
76
  | H100 throughput ↑ | **255×** real time | **243×** real time | 34× | 147× |
77
  | Laptop CPU, Q4 ↑ | **5.98×** | — | — | 3.06× |
78
  | Footprint, BF16 / Q4 | 1,447 / 721 MB | 3,830 / 1,642 MB | — | — |
79
 
80
+ The Q4 row and footprint are measured on the GGUF builds, where the Q4 CER figures are also
81
+ published (0.6B: 7.98 / 9.84%; 1.7B: 6.54 / 9.23%). Use those when you deploy Q4.
82
+
83
  ![Mean semantic error rate against Mistral Voxtral Mini 4B and MoulSot](comparison-ser.png)
84
 
85
  ![Accuracy against throughput on one H100](comparison-speed-accuracy.png)
 
88
  DarijaCS figures are its own published numbers; everything else is our rerun on the same
89
  clips. Lower is better for SER and CER, higher for throughput.
90
 
 
 
 
 
 
91
  ## Files
92
 
93
+ | File | Size | Description |
 
 
94
  |---|---|---|
95
+ | `model.safetensors` | 3.82 GB | Full BF16 weights (audio encoder + multimodal projector + language model). |
96
  | `config.json`, `generation_config.json`, `preprocessor_config.json` | — | Model, generation, and audio feature extractor configs. |
97
+ | `tokenizer.json`, `vocab.json`, `merges.txt`, `tokenizer_config.json`, `special_tokens_map.json`, `chat_template.json` | — | Shared 89k-token Darija tokenizer. |
98
+ | `sample.wav`, `sample.txt` | 97 KB | 3.11 s, 16 kHz mono DarijaCS Probe clip and its ground truth, for the reference check. |
99
 
100
+ ## Install
101
 
102
+ **GPU:**
 
103
 
104
  ```bash
105
  pip install -U qwen-asr
106
+ # or, with the vLLM backend:
107
  # pip install -U "qwen-asr[vllm]"
108
  ```
109
 
110
+ **CPU-only machine** — install the CPU torch wheel first so `accelerate` does not pull unused
111
+ CUDA libraries:
112
+
113
+ ```bash
114
+ pip install torch --index-url https://download.pytorch.org/whl/cpu
115
+ pip install -U qwen-asr
116
+ ```
117
+
118
+ `qwen-asr` 0.0.6 pins `transformers==4.57.6`, which does not include the `qwen3_asr`
119
+ architecture — `qwen-asr` registers it on import. Load the model through `qwen_asr` rather than
120
+ with bare `transformers`.
121
+
122
+ ## Quick start
123
 
124
  ```python
125
  import torch
 
136
 
137
  results = model.transcribe(
138
  audio="sample.wav",
139
+ language=None, # auto-detect; returns "Darija" for Darija audio
140
  )
141
 
142
  print(results[0].language)
143
  print(results[0].text)
144
  ```
145
 
146
+ `audio` accepts a local path, a URL, a base64 data URL, a `(np.ndarray, sample_rate)` tuple, or
147
+ a list of any of these. Audio should be 16 kHz mono; other rates are resampled.
148
+
149
+ ### Reading the output
150
+
151
+ The transcript is prefixed with the detected language, and the `<asr_text>` marker is
152
+ present or absent depending on the runtime. Parse both forms:
153
+
154
+ ```python
155
+ import re
156
+
157
+ ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)
158
+
159
+ def parse(raw: str) -> tuple[str | None, str]:
160
+ """Returns (language, transcript), with or without the <asr_text> marker."""
161
+ m = ASR_RE.search(raw.strip())
162
+ return (m.group(1), m.group(2).strip()) if m else (None, raw.strip())
163
+ ```
164
+
165
+ The language name is ASCII and runs directly into Arabic text, so match on `[A-Za-z]+` rather
166
+ than `\S+` — otherwise the first Arabic word is swallowed into the language field.
167
+
168
+ ## Manual `transformers` loop
169
+
170
+ For prompt control or batched decoding without `Qwen3ASRModel`:
171
+
172
+ ```python
173
+ import torch, soundfile as sf
174
+ import qwen_asr # registers qwen3_asr
175
+ from transformers import AutoProcessor, AutoModel
176
+
177
+ mid = "fihriaLab/fihrIA-ASR-1.7B"
178
+ processor = AutoProcessor.from_pretrained(mid)
179
+ model = AutoModel.from_pretrained(mid, dtype=torch.bfloat16, device_map="cuda:0")
180
+
181
+ wav, sr = sf.read("sample.wav") # 16 kHz mono
182
+ prompt = ("<|im_start|>system\n<|im_end|>\n"
183
+ "<|im_start|>user\n<|audio_start|><|audio_pad|><|audio_end|><|im_end|>\n"
184
+ "<|im_start|>assistant\n")
185
+
186
+ inputs = processor(text=prompt, audio=wav, sampling_rate=sr, return_tensors="pt")
187
+ inputs = inputs.to(model.device).to(model.dtype)
188
+ out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
189
+
190
+ new_tokens = out.sequences[:, inputs["input_ids"].shape[1]:]
191
+ raw = processor.batch_decode(new_tokens, skip_special_tokens=False)[0] # keeps <asr_text>
192
+ print(parse(raw))
193
+ ```
194
+
195
+ `skip_special_tokens=True` removes the `<asr_text>` marker, which is why the language label then
196
+ runs straight into the transcript.
197
+
198
+ ## vLLM backend
199
 
200
  ```python
 
201
  from qwen_asr import Qwen3ASRModel
202
 
203
  model = Qwen3ASRModel.LLM(
 
211
  print(results[0].text)
212
  ```
213
 
214
+ Requires `pip install "qwen-asr[vllm]"` (pins `vllm==0.14.0`) and a CUDA GPU.
215
+
216
  ## Prompt format
217
 
218
+ `qwen-asr` applies the canonical prompt automatically. For another runtime, use an empty
219
+ `system` turn and an **audio-only `user` turn** (no instruction text):
 
220
 
221
  ```
222
  <|im_start|>system\n<|im_end|>\n
 
224
  <|im_start|>assistant\n
225
  ```
226
 
227
+ Use greedy decoding (`temperature=0`) for reproducible output.
 
 
 
 
 
 
 
228
 
229
+ ## Output format
 
230
 
231
+ The decoder emits the detected language followed by `<asr_text>` and the transcript. Whether the
232
+ marker survives depends on the runtime:
233
 
234
+ | Runtime | Output shape |
235
+ |---|---|
236
+ | `qwen-asr`, auto-detect | `language Darija…` (marker may be absent) |
237
+ | `llama-mtmd-cli` | `language Darija<asr_text>التلفون…` |
238
+ | `llama-server` chat endpoint | `language Darijaالتلفون…` (no marker) |
239
 
240
+ Parse both forms — see [Reading the output](#reading-the-output).
 
241
 
242
  ## Reference output
243
 
244
+ `sample.wav` ships with this repo: 3.11 s, 16 kHz mono WAV, DarijaCS Probe, code-switched
245
+ Darija–French.
 
246
 
247
  Ground truth:
248
 
249
  <div dir="rtl" lang="ar">‏الtéléphone ديالي طاح وتهرس l'écran.</div>
250
 
251
+ Expected transcript after stripping the `language Darija` prefix and the `<asr_text>` marker:
252
 
253
  <div dir="rtl" lang="ar">‏التلفون ديالي طاح وتهرس ليه l'écran</div>
254
 
255
+ ```python
256
+ # end-to-end check
257
+ import re, torch
258
+ from qwen_asr import Qwen3ASRModel
259
+
260
+ ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)
261
+
262
+ model = Qwen3ASRModel.from_pretrained("fihriaLab/fihrIA-ASR-1.7B",
263
+ dtype=torch.bfloat16, device_map="cuda:0")
264
+ raw = model.transcribe(audio="sample.wav", language=None)[0].text
265
+ m = ASR_RE.search(raw.strip())
266
+ lang, text = (m.group(1), m.group(2).strip()) if m else (None, raw.strip())
267
+
268
+ assert lang == "Darija" and "طاح" in text, raw
269
+ print(lang, "|", text)
270
+ ```
271
+
272
  The model writes the French word in Arabic script (`الtéléphone` → `التلفون`) and adds `ليه`
273
+ before `l'écran`. The 0.6B runs through the same pipeline with its own weights.
274
 
275
  ## Checksums (MD5)
276
 
277
+ 1.7B:
278
 
279
  ```
280
+ e9e03c692d75413fbc94479e73d8c8ae model.safetensors
281
  ```
282
 
283
+ 0.6B:
284
 
285
  ```
286
+ 0c49e982df1c01adda76b886643f14e8 model.safetensors
287
  ```
288
 
289
+ `hf download` verifies these automatically; to check manually:
290
 
291
+ ```bash
292
+ hf download fihriaLab/fihrIA-ASR-1.7B --local-dir ./fihria-asr-1.7b
293
+ cd ./fihria-asr-1.7b && md5sum model.safetensors
294
+ ```
295
+
296
+ ## Citation
297
 
298
  ```bibtex
299
  @misc{fihria2026asr,
 
317
  - Vocal music is transcribed as lyrics, and very noisy or instrumental clips can occasionally
318
  produce short text. For noisy production pipelines, gating with a lightweight VAD (e.g.,
319
  Silero VAD) is recommended.
320
+ - Trained on Darija speech; performance on MSA, other Maghrebi dialects, and non-Arabic speech
321
+ is not characterised. Not evaluated for speaker attribution or emotion.
322
+ - Heavier than the 0.6B: budget ~8 GB of VRAM for BF16, and prefer the 0.6B where latency or
323
+ memory matter more than the last fraction of accuracy.
324
+
325
+ ## Contact
326
+
327
+ fihrIA is a Moroccan AI research lab working on Darija speech and language. Questions about
328
+ the model, the data, or a collaboration: **contact@fihria.ma**.