--- license: apache-2.0 language: - ar - ary pipeline_tag: automatic-speech-recognition library_name: transformers base_model: Qwen/Qwen3-ASR-0.6B tags: - automatic-speech-recognition - darija - fihria - moroccan-arabic - qwen3-asr - safetensors - code-switching --- # fihrIA-ASR-0.6B fihrIA-ASR is fihrIA lab's open-weight ASR model family purpose-built for Moroccan Darija. Based on Qwen3-ASR, trained using a shared 89k-token Darija tokenizer and evaluated under a unified protocol, the family is built for robust, real-world transcription: - **Robust code-switching:** seamlessly handles natural code-switching, accurately transcribing mixed sentences containing French and English words embedded within spoken Darija. - **Native Arabic script output:** direct transcription and formatting in Arabic script optimized for local linguistic structures. - **Tiered performance profile:** - **0.6B:** optimized for lightweight edge deployment and real-time execution on standard laptop CPUs (via the [GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF)). - **1.7B:** engineered as the high-accuracy ceiling and setting up a new SOTA for open-weight Darija ASR models. This repository holds the unquantized **BF16** weights. It is the GPU path; for CPU deployment use the [GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF). > **Try it live:** Test both 0.6B and 1.7B in your browser on the [fihrIA-ASR Demo Space](https://huggingface.co/spaces/fihria/fihrIA-ASR-Demo). ## Which build do I need? | Deployment | Build | Notes | |---|---|---| | CPU (laptop, edge, no GPU) | [GGUF Q4_K_M](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF) | Real-time on CPU; see the GGUF card | | NVIDIA GPU | **this repo** (BF16) | `qwen-asr`, transformers backend | | NVIDIA GPU, high throughput | **this repo** + vLLM | batching, `qwen-asr[vllm]` | **Memory:** 0.6B BF16 needs roughly 4 GB of VRAM (1.44 GB of weights plus KV cache and activations); 8 GB is comfortable for batching. The 1.7B needs about 8 GB. BF16 inference on CPU runs far below real time — use GGUF there. ## Technical & deployment details This repository hosts the unquantized **BF16 safetensors** weights, configs, and 89k Darija tokenizer. Inference runs through [`qwen-asr`](https://github.com/QwenLM/Qwen3-ASR), which provides both the `transformers` and `vLLM` backends and registers the `qwen3_asr` architecture with `transformers` on import. Repositories are hosted separately by size and format, sharing an identical layout: - **0.6B (safetensors):** [`fihriaLab/fihrIA-ASR-0.6B`](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B) - **1.7B (safetensors):** [`fihriaLab/fihrIA-ASR-1.7B`](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B) - **0.6B (GGUF / llama.cpp):** [`fihriaLab/fihrIA-ASR-0.6B-GGUF`](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF) - **1.7B (GGUF / llama.cpp):** [`fihriaLab/fihrIA-ASR-1.7B-GGUF`](https://huggingface.co/fihriaLab/fihrIA-ASR-1.7B-GGUF) - **Live Demo (Space):** [`fihria/fihrIA-ASR-Demo`](https://huggingface.co/spaces/fihria/fihrIA-ASR-Demo) ## Results **State of the art among open-weight Darija ASR models.** Both sizes beat every open-weight system we tested on held-out meaning error: the 1.7B reaches **6.1% SER** (vs 8.4% for Mistral's Voxtral Mini 4B and 11.2% for MoulSot) and achieves the best DarijaCS CER on both test sets, while running 7.1× faster than Voxtral on one H100. The 0.6B still beats Voxtral at **7.1% SER** in a laptop-sized footprint — 7.5× faster than Voxtral on H100 and 5.98× real time at 4-bit on a laptop CPU. BF16 weights (this repo): | | fihrIA-ASR 0.6B | fihrIA-ASR 1.7B | Mistral Voxtral Mini 4B | MoulSot 1.7B | |---|---|---|---|---| | Held-out mean SER ↓ | **7.1%** | **6.1%** | 8.4% | 11.2% | | DarijaCS CER ↓ (Prism / Probe, BF16) | **7.98 / 9.80%** | **6.24 / 8.71%** | 6.68 / 9.69% | 10.27 / 14.93% | | H100 throughput ↑ | **255×** real time | **243×** real time | 34× | 147× | | Laptop CPU, Q4 ↑ | **5.98×** | — | — | 3.06× | | Footprint, BF16 / Q4 | 1,447 / 721 MB | 3,830 / 1,642 MB | — | — | The Q4 row and footprint are measured on the GGUF builds, where the Q4 CER figures are also published (0.6B: 7.98 / 9.84%; 1.7B: 6.54 / 9.23%). Use those when you deploy Q4. ![Mean semantic error rate against Mistral Voxtral Mini 4B and MoulSot](comparison-ser.png) ![Accuracy against throughput on one H100](comparison-speed-accuracy.png) Held-out numbers are the mean of four independent LLM judges over 508 clips. Voxtral's DarijaCS figures are its own published numbers; everything else is our rerun on the same clips. Lower is better for SER and CER, higher for throughput. ## Files | File | Size (0.6B / 1.7B) | Description | |---|---|---| | `model.safetensors` | 1.44 GB / 3.82 GB | Full BF16 weights (audio encoder + multimodal projector + language model). | | `config.json`, `generation_config.json`, `preprocessor_config.json` | — | Model, generation, and audio feature extractor configs. | | `tokenizer.json`, `vocab.json`, `merges.txt`, `tokenizer_config.json`, `special_tokens_map.json`, `chat_template.json` | — | Shared 89k-token Darija tokenizer. | | `sample.wav` | 97 KB | 3.11 s, 16 kHz mono DarijaCS Probe clip, for the reference check. | | `sample.txt` | 55 B | Ground-truth transcript for `sample.wav`. | ## Install Requires Python 3.10–3.12 and a CUDA-capable NVIDIA GPU with a compatible driver. For CPU-only machines, use the [GGUF build](https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF) with `llama.cpp` instead — BF16 inference on CPU is impractical. ```bash pip install qwen-asr==0.0.6 # or, in a separate virtualenv for the vLLM backend: # pip install "qwen-asr[vllm]==0.0.6" ``` Budget **~6 GB on disk** for the virtualenv: `qwen-asr==0.0.6` pulls the default PyPI PyTorch CUDA build (CUDA 13.2 wheels + NVIDIA libraries; if your driver is on CUDA 12, install a matching PyTorch wheel from [pytorch.org](https://pytorch.org/get-started/locally/) first) along with `transformers==4.57.6` and UI/audio dependencies (`gradio`, `flask`, `pydub`, etc.). Note that `qwen-asr[vllm]==0.0.6` pins `vllm==0.14.0` (which pulls `torch==2.9.x` / CUDA 12 libraries), so use a fresh virtualenv per backend. Because `transformers==4.57.6` does not include the `qwen3_asr` architecture, `qwen-asr` registers it on import — load the model through `qwen_asr` rather than bare `transformers`. ## Quick start First fetch the bundled sample clip (or point `audio` at your own WAV/MP3/FLAC file); the first `from_pretrained()` call downloads ~1.44 GB of model weights: ```bash hf download fihriaLab/fihrIA-ASR-0.6B sample.wav --local-dir . ``` ```python import re, torch from qwen_asr import Qwen3ASRModel ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:)?\s*(.*)", re.DOTALL) model = Qwen3ASRModel.from_pretrained( "fihriaLab/fihrIA-ASR-0.6B", # or "fihriaLab/fihrIA-ASR-1.7B" dtype=torch.bfloat16, device_map="cuda:0", # attn_implementation="flash_attention_2", max_inference_batch_size=32, max_new_tokens=64, # ample for this 3 s clip; raise for longer audio (see EOS note below) ) res = model.transcribe( audio="sample.wav", language=None, # auto-detect; required for Darija (see note below) )[0] m = ASR_RE.search(res.text.strip()) lang, text = (m.group(1), m.group(2).strip()) if m else (None, res.text.strip()) print(lang) # Darija print(text) # التلفون ديالي طاح وتهرس ليه l'écran (exact form may vary slightly by runtime) ``` > **Notes for `qwen-asr==0.0.6`:** > - **Auto-detect output:** `res.language` is always empty (`""`) when `language=None`, because > `transcribe()` decodes with `skip_special_tokens=True` and strips the `` marker; > the language prefix (`language Darija`) is inside `res.text`. > - **Forcing Darija:** Always use `language=None` for Darija. `qwen-asr==0.0.6`'s built-in > `SUPPORTED_LANGUAGES` list predates Darija, so passing `language="Darija"` raises `ValueError` > (forcing other listed languages such as `"Arabic"` works and populates `res.language`). > - **`max_new_tokens` & EOS:** `transcribe()` decodes all `max_new_tokens` steps due to an > upstream EOS-id mismatch (see [Known limitations](#known-limitations)), so keeping > `max_new_tokens` tight for short clips avoids wasted decode steps. `audio` accepts a local path, a URL, a base64 data URL, a `(np.ndarray, sample_rate)` tuple, or a list of any of these. Audio should be 16 kHz mono; other rates are resampled. ### Reading the output The transcript is prefixed with the detected language, and the `` marker is present or absent depending on the runtime. Parse both forms: ```python import re ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:)?\s*(.*)", re.DOTALL) def parse(raw: str) -> tuple[str | None, str]: """Returns (language, transcript), with or without the marker.""" m = ASR_RE.search(raw.strip()) return (m.group(1), m.group(2).strip()) if m else (None, raw.strip()) ``` The language name is ASCII and runs directly into Arabic text, so match on `[A-Za-z]+` rather than `\S+` — otherwise the first Arabic word is swallowed into the language field. ## Manual `transformers` loop For prompt control or batched decoding without `Qwen3ASRModel` (self-contained): ```python import re, torch, soundfile as sf import qwen_asr # registers qwen3_asr from transformers import AutoProcessor, AutoModel ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:)?\s*(.*)", re.DOTALL) def parse(raw: str) -> tuple[str | None, str]: m = ASR_RE.search(raw.strip()) return (m.group(1), m.group(2).strip()) if m else (None, raw.strip()) mid = "fihriaLab/fihrIA-ASR-0.6B" processor = AutoProcessor.from_pretrained(mid) model = AutoModel.from_pretrained(mid, dtype=torch.bfloat16, device_map="cuda:0") wav, sr = sf.read("sample.wav") # 16 kHz mono prompt = ("<|im_start|>system\n<|im_end|>\n" "<|im_start|>user\n<|audio_start|><|audio_pad|><|audio_end|><|im_end|>\n" "<|im_start|>assistant\n") inputs = processor(text=prompt, audio=wav, sampling_rate=sr, return_tensors="pt") inputs = inputs.to(model.device).to(model.dtype) out = model.generate( **inputs, max_new_tokens=512, do_sample=False, eos_token_id=processor.tokenizer.eos_token_id, # 2 (<|im_end|>) for this model ) new_tokens = out.sequences[:, inputs["input_ids"].shape[1]:] raw = processor.batch_decode(new_tokens, skip_special_tokens=True)[0] print(parse(raw)) ``` Always pass `eos_token_id=processor.tokenizer.eos_token_id`: `qwen-asr==0.0.6` defaults `generate()` to Qwen2 EOS ids (`[151645, 151643]`), whereas this model's 89k tokenizer uses `<|im_end|> = 2`. Without it, generation will not stop at EOS and will pad out to `max_new_tokens` with repeated `<|im_end|>` tokens. `skip_special_tokens=True` strips both `<|im_end|>` and ``, and `parse()` handles the marker-absent output. ## vLLM backend ```python from qwen_asr import Qwen3ASRModel model = Qwen3ASRModel.LLM( model="fihriaLab/fihrIA-ASR-0.6B", # or "fihriaLab/fihrIA-ASR-1.7B" gpu_memory_utilization=0.7, max_inference_batch_size=128, max_new_tokens=512, ) results = model.transcribe(audio="sample.wav", language=None) print(results[0].text) ``` Requires `pip install "qwen-asr[vllm]==0.0.6"` (pins `vllm==0.14.0`) and a CUDA GPU. ## Prompt format `qwen-asr` applies the canonical prompt automatically. For another runtime, use an empty `system` turn and an **audio-only `user` turn** (no instruction text): ``` <|im_start|>system\n<|im_end|>\n <|im_start|>user\n<|audio_start|>[audio]<|audio_end|><|im_end|>\n <|im_start|>assistant\n ``` Use greedy decoding (`temperature=0`) for reproducible output. ## Output format The decoder emits the detected language followed by `` and the transcript. Whether the marker survives depends on the runtime: | Runtime | Output shape | |---|---| | `qwen-asr==0.0.6`, auto-detect | `language Darija…` (marker is always absent; `transcribe()` uses `skip_special_tokens=True`) | | `llama-mtmd-cli` | `language Darijaالتلفون…` | | `llama-server` chat endpoint | `language Darijaالتلفون…` (no marker) | Parse both forms — see [Reading the output](#reading-the-output). ## Reference output `sample.wav` ships with this repo: 3.11 s, 16 kHz mono WAV, DarijaCS Probe, code-switched Darija–French. Ground truth:
‏الtéléphone ديالي طاح وتهرس l'écran.
Expected transcript (exact form may vary slightly by runtime/precision, e.g. `التلفون` vs `تلفون`) after stripping the `language Darija` prefix and the `` marker:
‏التلفون ديالي طاح وتهرس ليه l'écran
```python # end-to-end check (assumes sample.wav has been downloaded, see Quick start) import re, torch from qwen_asr import Qwen3ASRModel ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:)?\s*(.*)", re.DOTALL) model = Qwen3ASRModel.from_pretrained("fihriaLab/fihrIA-ASR-0.6B", dtype=torch.bfloat16, device_map="cuda:0") raw = model.transcribe(audio="sample.wav", language=None)[0].text m = ASR_RE.search(raw.strip()) lang, text = (m.group(1), m.group(2).strip()) if m else (None, raw.strip()) assert lang == "Darija" and "طاح" in text, raw print(lang, "|", text) ``` The model writes the French word in Arabic script (`الtéléphone` → `التلفون` / `تلفون`) and adds `ليه` before `l'écran`. The 1.7B runs through the same pipeline with its own weights. ## Checksums (MD5) 0.6B: ``` 0c49e982df1c01adda76b886643f14e8 model.safetensors ``` 1.7B: ``` e9e03c692d75413fbc94479e73d8c8ae model.safetensors ``` Hugging Face verifies the LFS SHA-256 checksum during download. The MD5 values above are for an independent manual check: ```bash hf download fihriaLab/fihrIA-ASR-0.6B --local-dir ./fihria-asr-0.6b cd ./fihria-asr-0.6b && md5sum model.safetensors ``` ## Citation ```bibtex @misc{fihria2026asr, title = {fihrIA-ASR: Open-Weight Speech Recognition and Semantic Evaluation for Moroccan Darija}, author = {{fihrIA}}, year = {2026}, url = {https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B} } ``` ## Known limitations - Long audio is chunked in 30 s windows by the encoder; very long files are slower and may drift. - Fine-tuned for Darija. Other languages fall back to base Qwen3-ASR behaviour and are untested here. - Vocal music is transcribed as lyrics, and very noisy or instrumental clips can occasionally produce short text. For noisy production pipelines, gating with a lightweight VAD (e.g., Silero VAD) is recommended. - Trained on Darija speech; performance on MSA, other Maghrebi dialects, and non-Arabic speech is not characterised. Not evaluated for speaker attribution or emotion. - **`qwen-asr==0.0.6` EOS mismatch:** `Qwen3ASRModel.transcribe()` uses hardcoded Qwen2 EOS ids (`[151645, 151643]`) rather than this model's `<|im_end|> = 2`, and does not expose `eos_token_id`. As a result, `transcribe()` decodes all `max_new_tokens` steps (hiding trailing `<|im_end|>` tokens via `skip_special_tokens=True`). Set `max_new_tokens` to a realistic upper bound for your clips, or use the manual `transformers` loop above (with `eos_token_id=processor.tokenizer.eos_token_id`), vLLM, or the GGUF build. - CPU inference requires the GGUF builds; this BF16 repository is GPU-oriented. ## Contact fihrIA is a Moroccan AI research lab working on Darija speech and language. Questions about the model, the data, or a collaboration: **contact@fihria.ma**.