fihrIA-ASR-1.7B / README.md
fihria's picture
Add live demo Space link
2d32f24 verified
|
Raw History Blame Contribute Delete
16.1 kB
---
license: apache-2.0
language:
- ar
- ary
pipeline_tag: automatic-speech-recognition
library_name: transformers
base_model: Qwen/Qwen3-ASR-1.7B
tags:
- automatic-speech-recognition
- darija
- fihria
- moroccan-arabic
- qwen3-asr
- safetensors
- code-switching
---
# fihrIA-ASR-1.7B
fihrIA-ASR is fihrIA lab's open-weight ASR model family purpose-built for Moroccan Darija.
Based on Qwen3-ASR, trained using a shared 89k-token Darija tokenizer and evaluated under a
unified protocol, the family is built for robust, real-world transcription:
- **Robust code-switching:** seamlessly handles natural code-switching, accurately transcribing
mixed sentences containing French and English words embedded within spoken Darija.
- **Native Arabic script output:** direct transcription and formatting in Arabic script
optimized for local linguistic structures.
- **Tiered performance profile:**
- **0.6B:** optimized for lightweight edge deployment and real-time execution on standard
laptop CPUs (via the [GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF)).
- **1.7B:** engineered as the high-accuracy ceiling and setting up a new SOTA for
open-weight Darija ASR models.
This repository holds the unquantized **BF16** weights of the 1.7B, the accuracy ceiling of the
family. It is the GPU path; for CPU deployment use the
[GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF).
> **Try it live:** Test both 0.6B and 1.7B in your browser on the [fihrIA-ASR Demo Space](https://huggingface.co/spaces/fihria/fihrIA-ASR-Demo).
## Which build do I need?
| Deployment | Build | Notes |
|---|---|---|
| CPU (laptop, edge, no GPU) | [GGUF Q4_K_M](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF) | see the GGUF card |
| NVIDIA GPU | **this repo** (BF16) | `qwen-asr`, transformers backend |
| NVIDIA GPU, high throughput | **this repo** + vLLM | batching, `qwen-asr[vllm]` |
| Edge / real-time laptop CPU | [0.6B](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B) | smaller, faster, slightly higher SER |
**Memory:** 1.7B BF16 needs roughly 8 GB of VRAM (3.82 GB of weights plus KV cache and
activations). The 0.6B needs about 4 GB. BF16 inference on CPU runs far below real time — use
GGUF there.
## Technical & deployment details
This repository hosts the unquantized **BF16 safetensors** weights, configs, and 89k Darija
tokenizer. Inference runs through [`qwen-asr`](https://github.com/QwenLM/Qwen3-ASR), which
provides both the `transformers` and `vLLM` backends and registers the `qwen3_asr` architecture
with `transformers` on import.
Repositories are hosted separately by size and format, sharing an identical layout:
- **0.6B (safetensors):** [`fihriaLab/fihrIA-ASR-0.6B`](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B)
- **1.7B (safetensors):** [`fihriaLab/fihrIA-ASR-1.7B`](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B)
- **0.6B (GGUF / llama.cpp):** [`fihriaLab/fihrIA-ASR-0.6B-GGUF`](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF)
- **1.7B (GGUF / llama.cpp):** [`fihriaLab/fihrIA-ASR-1.7B-GGUF`](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF)
- **Live Demo (Space):** [`fihria/fihrIA-ASR-Demo`](https://huggingface.co/spaces/fihria/fihrIA-ASR-Demo)
## Results
**State of the art among open-weight Darija ASR models.** Both sizes beat every open-weight system we tested on held-out meaning error: the 1.7B reaches **6.1% SER** (vs 8.4% for Mistral's Voxtral Mini 4B and 11.2% for MoulSot) and achieves the best DarijaCS CER on both test sets, while running 7.1× faster than Voxtral on one H100. The 0.6B still beats Voxtral at **7.1% SER** in a laptop-sized footprint — 7.5× faster than Voxtral on H100 and 5.98× real time at 4-bit on a laptop CPU.
BF16 weights (this repo):
| | fihrIA-ASR 0.6B | fihrIA-ASR 1.7B | Mistral Voxtral Mini 4B | MoulSot 1.7B |
|---|---|---|---|---|
| Held-out mean SER ↓ | **7.1%** | **6.1%** | 8.4% | 11.2% |
| DarijaCS CER ↓ (Prism / Probe, BF16) | **7.98 / 9.80%** | **6.24 / 8.71%** | 6.68 / 9.69% | 10.27 / 14.93% |
| H100 throughput ↑ | **255×** real time | **243×** real time | 34× | 147× |
| Laptop CPU, Q4 ↑ | **5.98×** | — | — | 3.06× |
| Footprint, BF16 / Q4 | 1,447 / 721 MB | 3,830 / 1,642 MB | — | — |
The Q4 row and footprint are measured on the GGUF builds, where the Q4 CER figures are also
published (0.6B: 7.98 / 9.84%; 1.7B: 6.54 / 9.23%). Use those when you deploy Q4.
![Mean semantic error rate against Mistral Voxtral Mini 4B and MoulSot](comparison-ser.png)
![Accuracy against throughput on one H100](comparison-speed-accuracy.png)
Held-out numbers are the mean of four independent LLM judges over 508 clips. Voxtral's
DarijaCS figures are its own published numbers; everything else is our rerun on the same
clips. Lower is better for SER and CER, higher for throughput.
## Files
| File | Size | Description |
|---|---|---|
| `model.safetensors` | 3.82 GB | Full BF16 weights (audio encoder + multimodal projector + language model). |
| `config.json`, `generation_config.json`, `preprocessor_config.json` | — | Model, generation, and audio feature extractor configs. |
| `tokenizer.json`, `vocab.json`, `merges.txt`, `tokenizer_config.json`, `special_tokens_map.json`, `chat_template.json` | — | Shared 89k-token Darija tokenizer. |
| `sample.wav` | 97 KB | 3.11 s, 16 kHz mono DarijaCS Probe clip, for the reference check. |
| `sample.txt` | 55 B | Ground-truth transcript for `sample.wav`. |
## Install
Requires Python 3.10–3.12 and a CUDA-capable NVIDIA GPU with a compatible driver. For CPU-only
machines, use the [GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF) with
`llama.cpp` instead — BF16 inference on CPU is impractical.
```bash
pip install qwen-asr==0.0.6
# or, in a separate virtualenv for the vLLM backend:
# pip install "qwen-asr[vllm]==0.0.6"
```
Budget **~6 GB on disk** for the virtualenv: `qwen-asr==0.0.6` pulls the default PyPI PyTorch CUDA
build (CUDA 13.2 wheels + NVIDIA libraries; if your driver is on CUDA 12, install a matching
PyTorch wheel from [pytorch.org](https://pytorch.org/get-started/locally/) first) along with
`transformers==4.57.6` and UI/audio dependencies (`gradio`, `flask`, `pydub`, etc.). Note that
`qwen-asr[vllm]==0.0.6` pins `vllm==0.14.0` (which pulls `torch==2.9.x` / CUDA 12 libraries), so
use a fresh virtualenv per backend. Because `transformers==4.57.6` does not include the `qwen3_asr`
architecture, `qwen-asr` registers it on import — load the model through `qwen_asr` rather than
bare `transformers`.
## Quick start
First fetch the bundled sample clip (or point `audio` at your own WAV/MP3/FLAC file); the first
`from_pretrained()` call downloads ~3.82 GB of model weights:
```bash
hf download fihriaLab/fihrIA-ASR-1.7B sample.wav --local-dir .
```
```python
import re, torch
from qwen_asr import Qwen3ASRModel
ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)
model = Qwen3ASRModel.from_pretrained(
"fihriaLab/fihrIA-ASR-1.7B", # or "fihriaLab/fihrIA-ASR-0.6B"
dtype=torch.bfloat16,
device_map="cuda:0",
# attn_implementation="flash_attention_2",
max_inference_batch_size=32,
max_new_tokens=64, # ample for this 3 s clip; raise for longer audio (see EOS note below)
)
res = model.transcribe(
audio="sample.wav",
language=None, # auto-detect; required for Darija (see note below)
)[0]
m = ASR_RE.search(res.text.strip())
lang, text = (m.group(1), m.group(2).strip()) if m else (None, res.text.strip())
print(lang) # Darija
print(text) # التلفون ديالي طاح وتهرس ليه l'écran (exact form may vary slightly by runtime)
```
> **Notes for `qwen-asr==0.0.6`:**
> - **Auto-detect output:** `res.language` is always empty (`""`) when `language=None`, because
> `transcribe()` decodes with `skip_special_tokens=True` and strips the `<asr_text>` marker;
> the language prefix (`language Darija`) is inside `res.text`.
> - **Forcing Darija:** Always use `language=None` for Darija. `qwen-asr==0.0.6`'s built-in
> `SUPPORTED_LANGUAGES` list predates Darija, so passing `language="Darija"` raises `ValueError`
> (forcing other listed languages such as `"Arabic"` works and populates `res.language`).
> - **`max_new_tokens` & EOS:** `transcribe()` decodes all `max_new_tokens` steps due to an
> upstream EOS-id mismatch (see [Known limitations](#known-limitations)), so keeping
> `max_new_tokens` tight for short clips avoids wasted decode steps.
`audio` accepts a local path, a URL, a base64 data URL, a `(np.ndarray, sample_rate)` tuple, or
a list of any of these. Audio should be 16 kHz mono; other rates are resampled.
### Reading the output
The transcript is prefixed with the detected language, and the `<asr_text>` marker is
present or absent depending on the runtime. Parse both forms:
```python
import re
ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)
def parse(raw: str) -> tuple[str | None, str]:
"""Returns (language, transcript), with or without the <asr_text> marker."""
m = ASR_RE.search(raw.strip())
return (m.group(1), m.group(2).strip()) if m else (None, raw.strip())
```
The language name is ASCII and runs directly into Arabic text, so match on `[A-Za-z]+` rather
than `\S+` — otherwise the first Arabic word is swallowed into the language field.
## Manual `transformers` loop
For prompt control or batched decoding without `Qwen3ASRModel` (self-contained):
```python
import re, torch, soundfile as sf
import qwen_asr # registers qwen3_asr
from transformers import AutoProcessor, AutoModel
ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)
def parse(raw: str) -> tuple[str | None, str]:
m = ASR_RE.search(raw.strip())
return (m.group(1), m.group(2).strip()) if m else (None, raw.strip())
mid = "fihriaLab/fihrIA-ASR-1.7B"
processor = AutoProcessor.from_pretrained(mid)
model = AutoModel.from_pretrained(mid, dtype=torch.bfloat16, device_map="cuda:0")
wav, sr = sf.read("sample.wav") # 16 kHz mono
prompt = ("<|im_start|>system\n<|im_end|>\n"
"<|im_start|>user\n<|audio_start|><|audio_pad|><|audio_end|><|im_end|>\n"
"<|im_start|>assistant\n")
inputs = processor(text=prompt, audio=wav, sampling_rate=sr, return_tensors="pt")
inputs = inputs.to(model.device).to(model.dtype)
out = model.generate(
**inputs,
max_new_tokens=512,
do_sample=False,
eos_token_id=processor.tokenizer.eos_token_id, # 2 (<|im_end|>) for this model
)
new_tokens = out.sequences[:, inputs["input_ids"].shape[1]:]
raw = processor.batch_decode(new_tokens, skip_special_tokens=True)[0]
print(parse(raw))
```
Always pass `eos_token_id=processor.tokenizer.eos_token_id`: `qwen-asr==0.0.6` defaults
`generate()` to Qwen2 EOS ids (`[151645, 151643]`), whereas this model's 89k tokenizer uses
`<|im_end|> = 2`. Without it, generation will not stop at EOS and will pad out to `max_new_tokens`
with repeated `<|im_end|>` tokens. `skip_special_tokens=True` strips both `<|im_end|>` and
`<asr_text>`, and `parse()` handles the marker-absent output.
## vLLM backend
```python
from qwen_asr import Qwen3ASRModel
model = Qwen3ASRModel.LLM(
model="fihriaLab/fihrIA-ASR-1.7B", # or "fihriaLab/fihrIA-ASR-0.6B"
gpu_memory_utilization=0.7,
max_inference_batch_size=128,
max_new_tokens=512,
)
results = model.transcribe(audio="sample.wav", language=None)
print(results[0].text)
```
Requires `pip install "qwen-asr[vllm]==0.0.6"` (pins `vllm==0.14.0`) and a CUDA GPU.
## Prompt format
`qwen-asr` applies the canonical prompt automatically. For another runtime, use an empty
`system` turn and an **audio-only `user` turn** (no instruction text):
```
<|im_start|>system\n<|im_end|>\n
<|im_start|>user\n<|audio_start|>[audio]<|audio_end|><|im_end|>\n
<|im_start|>assistant\n
```
Use greedy decoding (`temperature=0`) for reproducible output.
## Output format
The decoder emits the detected language followed by `<asr_text>` and the transcript. Whether the
marker survives depends on the runtime:
| Runtime | Output shape |
|---|---|
| `qwen-asr==0.0.6`, auto-detect | `language Darija…` (marker is always absent; `transcribe()` uses `skip_special_tokens=True`) |
| `llama-mtmd-cli` | `language Darija<asr_text>التلفون…` |
| `llama-server` chat endpoint | `language Darijaالتلفون…` (no marker) |
Parse both forms — see [Reading the output](#reading-the-output).
## Reference output
`sample.wav` ships with this repo: 3.11 s, 16 kHz mono WAV, DarijaCS Probe, code-switched
Darija–French.
Ground truth:
<div dir="rtl" lang="ar">‏الtéléphone ديالي طاح وتهرس l'écran.</div>
Expected transcript (exact form may vary slightly by runtime/precision, e.g. `التلفون` vs
`تلفون`) after stripping the `language Darija` prefix and the `<asr_text>` marker:
<div dir="rtl" lang="ar">‏التلفون ديالي طاح وتهرس ليه l'écran</div>
```python
# end-to-end check (assumes sample.wav has been downloaded, see Quick start)
import re, torch
from qwen_asr import Qwen3ASRModel
ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)
model = Qwen3ASRModel.from_pretrained("fihriaLab/fihrIA-ASR-1.7B",
dtype=torch.bfloat16, device_map="cuda:0")
raw = model.transcribe(audio="sample.wav", language=None)[0].text
m = ASR_RE.search(raw.strip())
lang, text = (m.group(1), m.group(2).strip()) if m else (None, raw.strip())
assert lang == "Darija" and "طاح" in text, raw
print(lang, "|", text)
```
The model writes the French word in Arabic script (`الtéléphone` → `التلفون` / `تلفون`) and adds `ليه`
before `l'écran`. The 0.6B runs through the same pipeline with its own weights.
## Checksums (MD5)
1.7B:
```
e9e03c692d75413fbc94479e73d8c8ae model.safetensors
```
0.6B:
```
0c49e982df1c01adda76b886643f14e8 model.safetensors
```
Hugging Face verifies the LFS SHA-256 checksum during download. The MD5 values above are for an
independent manual check:
```bash
hf download fihriaLab/fihrIA-ASR-1.7B --local-dir ./fihria-asr-1.7b
cd ./fihria-asr-1.7b && md5sum model.safetensors
```
## Citation
```bibtex
@misc{fihria2026asr,
title = {fihrIA-ASR: Open-Weight Speech Recognition and Semantic Evaluation for Moroccan Darija},
author = {{fihrIA}},
year = {2026},
url = {https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B}
}
```
## Known limitations
- Long audio is chunked in 30 s windows by the encoder; very long files are slower and may drift.
- Fine-tuned for Darija. Other languages fall back to base Qwen3-ASR behaviour and are untested here.
- Vocal music is transcribed as lyrics, and very noisy or instrumental clips can occasionally
produce short text. For noisy production pipelines, gating with a lightweight VAD (e.g.,
Silero VAD) is recommended.
- Trained on Darija speech; performance on MSA, other Maghrebi dialects, and non-Arabic speech
is not characterised. Not evaluated for speaker attribution or emotion.
- **`qwen-asr==0.0.6` EOS mismatch:** `Qwen3ASRModel.transcribe()` uses hardcoded Qwen2 EOS ids
(`[151645, 151643]`) rather than this model's `<|im_end|> = 2`, and does not expose
`eos_token_id`. As a result, `transcribe()` decodes all `max_new_tokens` steps (hiding trailing
`<|im_end|>` tokens via `skip_special_tokens=True`). Set `max_new_tokens` to a realistic upper
bound for your clips, or use the manual `transformers` loop above (with
`eos_token_id=processor.tokenizer.eos_token_id`), vLLM, or the GGUF build.
- Heavier than the 0.6B: budget ~8 GB of VRAM for BF16, and prefer the 0.6B where latency or
memory matter more than the last fraction of accuracy.
- CPU inference requires the GGUF builds; this BF16 repository is GPU-oriented.
## Contact
fihrIA is a Moroccan AI research lab working on Darija speech and language. Questions about
the model, the data, or a collaboration: **contact@fihria.ma**.