Automatic Speech Recognition
Transformers
Safetensors
Arabic
Moroccan Arabic
qwen3_asr
darija
fihria
moroccan-arabic
qwen3-asr
code-switching
Instructions to use fihriaLab/fihrIA-ASR-1.7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use fihriaLab/fihrIA-ASR-1.7B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="fihriaLab/fihrIA-ASR-1.7B")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("fihriaLab/fihrIA-ASR-1.7B") model = AutoModelForMultimodalLM.from_pretrained("fihriaLab/fihrIA-ASR-1.7B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from fihriaLab/fihrIA-ASR-1.7B: direct link, hf CLI and curl.
- Browser
- Download file 16.1 kB
-
https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B/resolve/main/README.md
- Command line
-
hf download hf://fihriaLab/fihrIA-ASR-1.7B/README.md
-
curl -L -o README.md https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B/resolve/main/README.md
16.1 kB
| license: apache-2.0 | |
| language: | |
| - ar | |
| - ary | |
| pipeline_tag: automatic-speech-recognition | |
| library_name: transformers | |
| base_model: Qwen/Qwen3-ASR-1.7B | |
| tags: | |
| - automatic-speech-recognition | |
| - darija | |
| - fihria | |
| - moroccan-arabic | |
| - qwen3-asr | |
| - safetensors | |
| - code-switching | |
| # fihrIA-ASR-1.7B | |
| fihrIA-ASR is fihrIA lab's open-weight ASR model family purpose-built for Moroccan Darija. | |
| Based on Qwen3-ASR, trained using a shared 89k-token Darija tokenizer and evaluated under a | |
| unified protocol, the family is built for robust, real-world transcription: | |
| - **Robust code-switching:** seamlessly handles natural code-switching, accurately transcribing | |
| mixed sentences containing French and English words embedded within spoken Darija. | |
| - **Native Arabic script output:** direct transcription and formatting in Arabic script | |
| optimized for local linguistic structures. | |
| - **Tiered performance profile:** | |
| - **0.6B:** optimized for lightweight edge deployment and real-time execution on standard | |
| laptop CPUs (via the [GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF)). | |
| - **1.7B:** engineered as the high-accuracy ceiling and setting up a new SOTA for | |
| open-weight Darija ASR models. | |
| This repository holds the unquantized **BF16** weights of the 1.7B, the accuracy ceiling of the | |
| family. It is the GPU path; for CPU deployment use the | |
| [GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF). | |
| > **Try it live:** Test both 0.6B and 1.7B in your browser on the [fihrIA-ASR Demo Space](https://huggingface.co/spaces/fihria/fihrIA-ASR-Demo). | |
| ## Which build do I need? | |
| | Deployment | Build | Notes | | |
| |---|---|---| | |
| | CPU (laptop, edge, no GPU) | [GGUF Q4_K_M](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF) | see the GGUF card | | |
| | NVIDIA GPU | **this repo** (BF16) | `qwen-asr`, transformers backend | | |
| | NVIDIA GPU, high throughput | **this repo** + vLLM | batching, `qwen-asr[vllm]` | | |
| | Edge / real-time laptop CPU | [0.6B](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B) | smaller, faster, slightly higher SER | | |
| **Memory:** 1.7B BF16 needs roughly 8 GB of VRAM (3.82 GB of weights plus KV cache and | |
| activations). The 0.6B needs about 4 GB. BF16 inference on CPU runs far below real time — use | |
| GGUF there. | |
| ## Technical & deployment details | |
| This repository hosts the unquantized **BF16 safetensors** weights, configs, and 89k Darija | |
| tokenizer. Inference runs through [`qwen-asr`](https://github.com/QwenLM/Qwen3-ASR), which | |
| provides both the `transformers` and `vLLM` backends and registers the `qwen3_asr` architecture | |
| with `transformers` on import. | |
| Repositories are hosted separately by size and format, sharing an identical layout: | |
| - **0.6B (safetensors):** [`fihriaLab/fihrIA-ASR-0.6B`](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B) | |
| - **1.7B (safetensors):** [`fihriaLab/fihrIA-ASR-1.7B`](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B) | |
| - **0.6B (GGUF / llama.cpp):** [`fihriaLab/fihrIA-ASR-0.6B-GGUF`](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF) | |
| - **1.7B (GGUF / llama.cpp):** [`fihriaLab/fihrIA-ASR-1.7B-GGUF`](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF) | |
| - **Live Demo (Space):** [`fihria/fihrIA-ASR-Demo`](https://huggingface.co/spaces/fihria/fihrIA-ASR-Demo) | |
| ## Results | |
| **State of the art among open-weight Darija ASR models.** Both sizes beat every open-weight system we tested on held-out meaning error: the 1.7B reaches **6.1% SER** (vs 8.4% for Mistral's Voxtral Mini 4B and 11.2% for MoulSot) and achieves the best DarijaCS CER on both test sets, while running 7.1× faster than Voxtral on one H100. The 0.6B still beats Voxtral at **7.1% SER** in a laptop-sized footprint — 7.5× faster than Voxtral on H100 and 5.98× real time at 4-bit on a laptop CPU. | |
| BF16 weights (this repo): | |
| | | fihrIA-ASR 0.6B | fihrIA-ASR 1.7B | Mistral Voxtral Mini 4B | MoulSot 1.7B | | |
| |---|---|---|---|---| | |
| | Held-out mean SER ↓ | **7.1%** | **6.1%** | 8.4% | 11.2% | | |
| | DarijaCS CER ↓ (Prism / Probe, BF16) | **7.98 / 9.80%** | **6.24 / 8.71%** | 6.68 / 9.69% | 10.27 / 14.93% | | |
| | H100 throughput ↑ | **255×** real time | **243×** real time | 34× | 147× | | |
| | Laptop CPU, Q4 ↑ | **5.98×** | — | — | 3.06× | | |
| | Footprint, BF16 / Q4 | 1,447 / 721 MB | 3,830 / 1,642 MB | — | — | | |
| The Q4 row and footprint are measured on the GGUF builds, where the Q4 CER figures are also | |
| published (0.6B: 7.98 / 9.84%; 1.7B: 6.54 / 9.23%). Use those when you deploy Q4. | |
|  | |
|  | |
| Held-out numbers are the mean of four independent LLM judges over 508 clips. Voxtral's | |
| DarijaCS figures are its own published numbers; everything else is our rerun on the same | |
| clips. Lower is better for SER and CER, higher for throughput. | |
| ## Files | |
| | File | Size | Description | | |
| |---|---|---| | |
| | `model.safetensors` | 3.82 GB | Full BF16 weights (audio encoder + multimodal projector + language model). | | |
| | `config.json`, `generation_config.json`, `preprocessor_config.json` | — | Model, generation, and audio feature extractor configs. | | |
| | `tokenizer.json`, `vocab.json`, `merges.txt`, `tokenizer_config.json`, `special_tokens_map.json`, `chat_template.json` | — | Shared 89k-token Darija tokenizer. | | |
| | `sample.wav` | 97 KB | 3.11 s, 16 kHz mono DarijaCS Probe clip, for the reference check. | | |
| | `sample.txt` | 55 B | Ground-truth transcript for `sample.wav`. | | |
| ## Install | |
| Requires Python 3.10–3.12 and a CUDA-capable NVIDIA GPU with a compatible driver. For CPU-only | |
| machines, use the [GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF) with | |
| `llama.cpp` instead — BF16 inference on CPU is impractical. | |
| ```bash | |
| pip install qwen-asr==0.0.6 | |
| # or, in a separate virtualenv for the vLLM backend: | |
| # pip install "qwen-asr[vllm]==0.0.6" | |
| ``` | |
| Budget **~6 GB on disk** for the virtualenv: `qwen-asr==0.0.6` pulls the default PyPI PyTorch CUDA | |
| build (CUDA 13.2 wheels + NVIDIA libraries; if your driver is on CUDA 12, install a matching | |
| PyTorch wheel from [pytorch.org](https://pytorch.org/get-started/locally/) first) along with | |
| `transformers==4.57.6` and UI/audio dependencies (`gradio`, `flask`, `pydub`, etc.). Note that | |
| `qwen-asr[vllm]==0.0.6` pins `vllm==0.14.0` (which pulls `torch==2.9.x` / CUDA 12 libraries), so | |
| use a fresh virtualenv per backend. Because `transformers==4.57.6` does not include the `qwen3_asr` | |
| architecture, `qwen-asr` registers it on import — load the model through `qwen_asr` rather than | |
| bare `transformers`. | |
| ## Quick start | |
| First fetch the bundled sample clip (or point `audio` at your own WAV/MP3/FLAC file); the first | |
| `from_pretrained()` call downloads ~3.82 GB of model weights: | |
| ```bash | |
| hf download fihriaLab/fihrIA-ASR-1.7B sample.wav --local-dir . | |
| ``` | |
| ```python | |
| import re, torch | |
| from qwen_asr import Qwen3ASRModel | |
| ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL) | |
| model = Qwen3ASRModel.from_pretrained( | |
| "fihriaLab/fihrIA-ASR-1.7B", # or "fihriaLab/fihrIA-ASR-0.6B" | |
| dtype=torch.bfloat16, | |
| device_map="cuda:0", | |
| # attn_implementation="flash_attention_2", | |
| max_inference_batch_size=32, | |
| max_new_tokens=64, # ample for this 3 s clip; raise for longer audio (see EOS note below) | |
| ) | |
| res = model.transcribe( | |
| audio="sample.wav", | |
| language=None, # auto-detect; required for Darija (see note below) | |
| )[0] | |
| m = ASR_RE.search(res.text.strip()) | |
| lang, text = (m.group(1), m.group(2).strip()) if m else (None, res.text.strip()) | |
| print(lang) # Darija | |
| print(text) # التلفون ديالي طاح وتهرس ليه l'écran (exact form may vary slightly by runtime) | |
| ``` | |
| > **Notes for `qwen-asr==0.0.6`:** | |
| > - **Auto-detect output:** `res.language` is always empty (`""`) when `language=None`, because | |
| > `transcribe()` decodes with `skip_special_tokens=True` and strips the `<asr_text>` marker; | |
| > the language prefix (`language Darija`) is inside `res.text`. | |
| > - **Forcing Darija:** Always use `language=None` for Darija. `qwen-asr==0.0.6`'s built-in | |
| > `SUPPORTED_LANGUAGES` list predates Darija, so passing `language="Darija"` raises `ValueError` | |
| > (forcing other listed languages such as `"Arabic"` works and populates `res.language`). | |
| > - **`max_new_tokens` & EOS:** `transcribe()` decodes all `max_new_tokens` steps due to an | |
| > upstream EOS-id mismatch (see [Known limitations](#known-limitations)), so keeping | |
| > `max_new_tokens` tight for short clips avoids wasted decode steps. | |
| `audio` accepts a local path, a URL, a base64 data URL, a `(np.ndarray, sample_rate)` tuple, or | |
| a list of any of these. Audio should be 16 kHz mono; other rates are resampled. | |
| ### Reading the output | |
| The transcript is prefixed with the detected language, and the `<asr_text>` marker is | |
| present or absent depending on the runtime. Parse both forms: | |
| ```python | |
| import re | |
| ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL) | |
| def parse(raw: str) -> tuple[str | None, str]: | |
| """Returns (language, transcript), with or without the <asr_text> marker.""" | |
| m = ASR_RE.search(raw.strip()) | |
| return (m.group(1), m.group(2).strip()) if m else (None, raw.strip()) | |
| ``` | |
| The language name is ASCII and runs directly into Arabic text, so match on `[A-Za-z]+` rather | |
| than `\S+` — otherwise the first Arabic word is swallowed into the language field. | |
| ## Manual `transformers` loop | |
| For prompt control or batched decoding without `Qwen3ASRModel` (self-contained): | |
| ```python | |
| import re, torch, soundfile as sf | |
| import qwen_asr # registers qwen3_asr | |
| from transformers import AutoProcessor, AutoModel | |
| ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL) | |
| def parse(raw: str) -> tuple[str | None, str]: | |
| m = ASR_RE.search(raw.strip()) | |
| return (m.group(1), m.group(2).strip()) if m else (None, raw.strip()) | |
| mid = "fihriaLab/fihrIA-ASR-1.7B" | |
| processor = AutoProcessor.from_pretrained(mid) | |
| model = AutoModel.from_pretrained(mid, dtype=torch.bfloat16, device_map="cuda:0") | |
| wav, sr = sf.read("sample.wav") # 16 kHz mono | |
| prompt = ("<|im_start|>system\n<|im_end|>\n" | |
| "<|im_start|>user\n<|audio_start|><|audio_pad|><|audio_end|><|im_end|>\n" | |
| "<|im_start|>assistant\n") | |
| inputs = processor(text=prompt, audio=wav, sampling_rate=sr, return_tensors="pt") | |
| inputs = inputs.to(model.device).to(model.dtype) | |
| out = model.generate( | |
| **inputs, | |
| max_new_tokens=512, | |
| do_sample=False, | |
| eos_token_id=processor.tokenizer.eos_token_id, # 2 (<|im_end|>) for this model | |
| ) | |
| new_tokens = out.sequences[:, inputs["input_ids"].shape[1]:] | |
| raw = processor.batch_decode(new_tokens, skip_special_tokens=True)[0] | |
| print(parse(raw)) | |
| ``` | |
| Always pass `eos_token_id=processor.tokenizer.eos_token_id`: `qwen-asr==0.0.6` defaults | |
| `generate()` to Qwen2 EOS ids (`[151645, 151643]`), whereas this model's 89k tokenizer uses | |
| `<|im_end|> = 2`. Without it, generation will not stop at EOS and will pad out to `max_new_tokens` | |
| with repeated `<|im_end|>` tokens. `skip_special_tokens=True` strips both `<|im_end|>` and | |
| `<asr_text>`, and `parse()` handles the marker-absent output. | |
| ## vLLM backend | |
| ```python | |
| from qwen_asr import Qwen3ASRModel | |
| model = Qwen3ASRModel.LLM( | |
| model="fihriaLab/fihrIA-ASR-1.7B", # or "fihriaLab/fihrIA-ASR-0.6B" | |
| gpu_memory_utilization=0.7, | |
| max_inference_batch_size=128, | |
| max_new_tokens=512, | |
| ) | |
| results = model.transcribe(audio="sample.wav", language=None) | |
| print(results[0].text) | |
| ``` | |
| Requires `pip install "qwen-asr[vllm]==0.0.6"` (pins `vllm==0.14.0`) and a CUDA GPU. | |
| ## Prompt format | |
| `qwen-asr` applies the canonical prompt automatically. For another runtime, use an empty | |
| `system` turn and an **audio-only `user` turn** (no instruction text): | |
| ``` | |
| <|im_start|>system\n<|im_end|>\n | |
| <|im_start|>user\n<|audio_start|>[audio]<|audio_end|><|im_end|>\n | |
| <|im_start|>assistant\n | |
| ``` | |
| Use greedy decoding (`temperature=0`) for reproducible output. | |
| ## Output format | |
| The decoder emits the detected language followed by `<asr_text>` and the transcript. Whether the | |
| marker survives depends on the runtime: | |
| | Runtime | Output shape | | |
| |---|---| | |
| | `qwen-asr==0.0.6`, auto-detect | `language Darija…` (marker is always absent; `transcribe()` uses `skip_special_tokens=True`) | | |
| | `llama-mtmd-cli` | `language Darija<asr_text>التلفون…` | | |
| | `llama-server` chat endpoint | `language Darijaالتلفون…` (no marker) | | |
| Parse both forms — see [Reading the output](#reading-the-output). | |
| ## Reference output | |
| `sample.wav` ships with this repo: 3.11 s, 16 kHz mono WAV, DarijaCS Probe, code-switched | |
| Darija–French. | |
| Ground truth: | |
| <div dir="rtl" lang="ar">الtéléphone ديالي طاح وتهرس l'écran.</div> | |
| Expected transcript (exact form may vary slightly by runtime/precision, e.g. `التلفون` vs | |
| `تلفون`) after stripping the `language Darija` prefix and the `<asr_text>` marker: | |
| <div dir="rtl" lang="ar">التلفون ديالي طاح وتهرس ليه l'écran</div> | |
| ```python | |
| # end-to-end check (assumes sample.wav has been downloaded, see Quick start) | |
| import re, torch | |
| from qwen_asr import Qwen3ASRModel | |
| ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL) | |
| model = Qwen3ASRModel.from_pretrained("fihriaLab/fihrIA-ASR-1.7B", | |
| dtype=torch.bfloat16, device_map="cuda:0") | |
| raw = model.transcribe(audio="sample.wav", language=None)[0].text | |
| m = ASR_RE.search(raw.strip()) | |
| lang, text = (m.group(1), m.group(2).strip()) if m else (None, raw.strip()) | |
| assert lang == "Darija" and "طاح" in text, raw | |
| print(lang, "|", text) | |
| ``` | |
| The model writes the French word in Arabic script (`الtéléphone` → `التلفون` / `تلفون`) and adds `ليه` | |
| before `l'écran`. The 0.6B runs through the same pipeline with its own weights. | |
| ## Checksums (MD5) | |
| 1.7B: | |
| ``` | |
| e9e03c692d75413fbc94479e73d8c8ae model.safetensors | |
| ``` | |
| 0.6B: | |
| ``` | |
| 0c49e982df1c01adda76b886643f14e8 model.safetensors | |
| ``` | |
| Hugging Face verifies the LFS SHA-256 checksum during download. The MD5 values above are for an | |
| independent manual check: | |
| ```bash | |
| hf download fihriaLab/fihrIA-ASR-1.7B --local-dir ./fihria-asr-1.7b | |
| cd ./fihria-asr-1.7b && md5sum model.safetensors | |
| ``` | |
| ## Citation | |
| ```bibtex | |
| @misc{fihria2026asr, | |
| title = {fihrIA-ASR: Open-Weight Speech Recognition and Semantic Evaluation for Moroccan Darija}, | |
| author = {{fihrIA}}, | |
| year = {2026}, | |
| url = {https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B} | |
| } | |
| ``` | |
| ## Known limitations | |
| - Long audio is chunked in 30 s windows by the encoder; very long files are slower and may drift. | |
| - Fine-tuned for Darija. Other languages fall back to base Qwen3-ASR behaviour and are untested here. | |
| - Vocal music is transcribed as lyrics, and very noisy or instrumental clips can occasionally | |
| produce short text. For noisy production pipelines, gating with a lightweight VAD (e.g., | |
| Silero VAD) is recommended. | |
| - Trained on Darija speech; performance on MSA, other Maghrebi dialects, and non-Arabic speech | |
| is not characterised. Not evaluated for speaker attribution or emotion. | |
| - **`qwen-asr==0.0.6` EOS mismatch:** `Qwen3ASRModel.transcribe()` uses hardcoded Qwen2 EOS ids | |
| (`[151645, 151643]`) rather than this model's `<|im_end|> = 2`, and does not expose | |
| `eos_token_id`. As a result, `transcribe()` decodes all `max_new_tokens` steps (hiding trailing | |
| `<|im_end|>` tokens via `skip_special_tokens=True`). Set `max_new_tokens` to a realistic upper | |
| bound for your clips, or use the manual `transformers` loop above (with | |
| `eos_token_id=processor.tokenizer.eos_token_id`), vLLM, or the GGUF build. | |
| - Heavier than the 0.6B: budget ~8 GB of VRAM for BF16, and prefer the 0.6B where latency or | |
| memory matter more than the last fraction of accuracy. | |
| - CPU inference requires the GGUF builds; this BF16 repository is GPU-oriented. | |
| ## Contact | |
| fihrIA is a Moroccan AI research lab working on Darija speech and language. Questions about | |
| the model, the data, or a collaboration: **contact@fihria.ma**. | |