Instructions to use fihriaLab/fihrIA-ASR-0.6B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use fihriaLab/fihrIA-ASR-0.6B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16 # Run inference directly in the terminal: llama cli -hf fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16 # Run inference directly in the terminal: llama cli -hf fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16 # Run inference directly in the terminal: ./build/bin/llama cli -hf fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16
Use Docker
docker model run hf.co/fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16
- Llama new
- LM Studio
- Jan
- Ollama
How to use fihriaLab/fihrIA-ASR-0.6B-GGUF with Ollama:
ollama run hf.co/fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16
- Unsloth Desktop
- Docker Model Runner
How to use fihriaLab/fihrIA-ASR-0.6B-GGUF with Docker Model Runner:
docker model run hf.co/fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16
- Lemonade
How to use fihriaLab/fihrIA-ASR-0.6B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull fihriaLab/fihrIA-ASR-0.6B-GGUF:BF16
Run and chat with the model
lemonade run user.fihrIA-ASR-0.6B-GGUF-BF16
List all available models
lemonade list
- Atomic Chat
fihrIA-ASR-0.6B โ GGUF
fihrIA-ASR is fihrIA lab's open-weight ASR model family purpose-built for Moroccan Darija. Based on Qwen3-ASR, trained using a shared 89k-token Darija tokenizer and evaluated under a unified protocol, the family is built for robust, real-world transcription:
- Robust code-switching: seamlessly handles natural code-switching, accurately transcribing mixed sentences containing French and English words embedded within spoken Darija.
- Native Arabic script output: direct transcription and formatting in Arabic script optimized for local linguistic structures.
- Tiered performance profile:
- 0.6B: optimized for lightweight edge deployment and real-time execution on standard laptop CPUs.
- 1.7B: engineered as the high-accuracy ceiling and setting up a new SOTA for open-weight Darija ASR models.
These GGUF builds are the CPU path for this model family. For GPU, use the BF16 safetensors repositories.
Try it live: Test both 0.6B and 1.7B in your browser on the fihrIA-ASR Demo Space.
Which build do I need?
| Deployment | Build | Notes |
|---|---|---|
| CPU (laptop, edge) | Q4_K_M + mmproj (this repo) | real-time on CPU |
| CPU, maximum quality | BF16 GGUF + mmproj | same layout, larger files |
| NVIDIA GPU | BF16 safetensors | qwen-asr, transformers backend |
| NVIDIA GPU, high throughput | BF16 safetensors + vLLM | batching |
Memory:
| Build | Files on disk | RAM at -c 8192 |
|---|---|---|
| 0.6B Q4_K_M | 342 MB + 379 MB mmproj | ~1.6 GB |
| 0.6B BF16 GGUF | 1,068 MB + 379 MB mmproj | ~2.4 GB |
| 1.7B Q4_K_M | 1,000 MB + 642 MB mmproj | ~2.6 GB |
RAM includes the ~900 MB KV cache at -c 8192; lowering -c reduces it (see
Settings that affect accuracy).
Technical & deployment details
These builds are packaged for llama.cpp via its
multimodal (mtmd) stack. Audio processing needs both language modeling and feature
projection, so each size ships as two components: the language model and its corresponding
mmproj audio projector.
Repositories are hosted separately by size and format, sharing an identical layout:
- 0.6B (safetensors):
fihriaLab/fihrIA-ASR-0.6B - 1.7B (safetensors):
fihriaLab/fihrIA-ASR-1.7B - 0.6B (GGUF / llama.cpp):
fihriaLab/fihrIA-ASR-0.6B-GGUF - 1.7B (GGUF / llama.cpp):
fihriaLab/fihrIA-ASR-1.7B-GGUF - Live Demo (Space):
fihria/fihrIA-ASR-Demo
Results
State of the art among open-weight Darija ASR models. Both sizes beat every open-weight system we tested on held-out meaning error: the 1.7B reaches 6.1% SER (vs 8.4% for Mistral's Voxtral Mini 4B and 11.2% for MoulSot) and achieves the best DarijaCS CER on both test sets, while running 7.1ร faster than Voxtral on one H100. The 0.6B still beats Voxtral at 7.1% SER in a laptop-sized footprint โ 7.5ร faster than Voxtral on H100 and 5.98ร real time at 4-bit on a laptop CPU.
Q4_K_M builds (this repo):
| fihrIA-ASR 0.6B | fihrIA-ASR 1.7B | Mistral Voxtral Mini 4B | MoulSot 1.7B | |
|---|---|---|---|---|
| Held-out mean SER โ | 7.1% | 6.1% | 8.4% | 11.2% |
| DarijaCS CER โ (Prism / Probe, Q4) | 7.98 / 9.84% | 6.54 / 9.23% | 6.68 / 9.69% | 10.27 / 14.93% |
| H100 throughput โ | 255ร real time | 243ร real time | 34ร | 147ร |
| Laptop CPU, Q4 โ | 5.98ร | โ | โ | 3.06ร |
| Footprint, BF16 / Q4 | 1,447 / 721 MB | 3,830 / 1,642 MB | โ | โ |
The safetensors repositories report BF16 CER (0.6B: 7.98 / 9.80%; 1.7B: 6.24 / 8.71%). The figures above are the Q4 ones, and are the ones to compare against when you deploy Q4.
Held-out numbers are the mean of four independent LLM judges over 508 clips. Voxtral's DarijaCS figures are its own published numbers; everything else is our rerun on the same clips. Lower is better for SER and CER, higher for throughput.
Files
Filenames follow the size:
| File | Size | Description |
|---|---|---|
fihria-asr-0.6b-Q4_K_M.gguf |
342 MB | 0.6B language model, 4-bit โ the configuration used for the published evaluation. |
fihria-asr-0.6b-BF16.gguf |
1,068 MB | 0.6B language model, full precision. |
mmproj-fihria-asr-0.6b-BF16.gguf |
379 MB | 0.6B audio projector. Always required. |
fihria-asr-1.7b-Q4_K_M.gguf |
1,000 MB | 1.7B language model, 4-bit. |
fihria-asr-1.7b-BF16.gguf |
3,188 MB | 1.7B language model, full precision. |
mmproj-fihria-asr-1.7b-BF16.gguf |
642 MB | 1.7B audio projector. Always required. |
sample.wav |
97 KB | 3.11 s, 16 kHz mono DarijaCS Probe clip, for the reference check. |
transcribe.py |
6 KB | Stdlib-only client for llama-server. |
Always pair the language model with its own mmproj file โ the 0.6B and 1.7B audio towers
are not interchangeable.
Keep the mmproj in BF16. A quantized audio encoder saves little memory and has not been
validated for this model.
Build llama.cpp
Use build 9199 (39cf5d619, 2026-05-17) or newer. Older builds load these files without
error and produce fluent-looking output with silently wrong words.
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 39cf5d619 # or any newer commit
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-server llama-mtmd-cli
./build/bin/llama-mtmd-cli --version # expect: 9199 (39cf5d619) or newer
A CPU-only build is fine; the reference results were produced on CPU.
The build fetches web-UI assets for llama-server. On an offline or proxied machine that step
can fail with xxd.cmake: file failed to open ... bundle.css; it is a missing download, not a
bad commit. llama-mtmd-cli builds without the UI and is enough for transcription:
cmake --build build -j --target llama-mtmd-cli
Quick start
Download the two files you need:
# hf CLI: pip install -U huggingface_hub
hf download fihriaLab/fihrIA-ASR-0.6B-GGUF \
fihria-asr-0.6b-Q4_K_M.gguf \
mmproj-fihria-asr-0.6b-BF16.gguf \
transcribe.py \
sample.wav \
--local-dir ./fihria-asr-0.6b
# 1.7B
hf download fihriaLab/fihrIA-ASR-1.7B-GGUF \
fihria-asr-1.7b-Q4_K_M.gguf \
mmproj-fihria-asr-1.7b-BF16.gguf \
--local-dir ./fihria-asr-1.7b
The examples below use the 0.6B filenames; swap 0.6b for 1.7b to run the 1.7B.
One-shot transcription
llama-mtmd-cli \
-m fihria-asr-0.6b-Q4_K_M.gguf \
--mmproj mmproj-fihria-asr-0.6b-BF16.gguf \
--audio sample.wav \
-p " " \
--jinja -c 8192 -n 512 --temp 0
Keep -p " " as written. An empty -p "" is read as "no prompt" and starts interactive chat
mode; a single space keeps one-shot mode without adding instruction text to the audio-only
user turn.
Expected output (CPU, comfortably faster than real time):
language Darija<asr_text>ุงูุชูููู ุฏูุงูู ุทุงุญ ูุชูุฑุณ ููู l'รฉcran
Server
llama-server \
-m fihria-asr-0.6b-Q4_K_M.gguf \
--mmproj mmproj-fihria-asr-0.6b-BF16.gguf \
--jinja --no-mmproj-offload -ngl 0 -c 8192 \
--host 127.0.0.1 --port 8080
Send audio as base64 through the OpenAI-compatible endpoint. A ready-made client is included as
transcribe.py:
python3 transcribe.py --audio sample.wav
python3 transcribe.py --input-jsonl clips.jsonl --output results.jsonl
Or with curl:
AUDIO_B64=$(base64 -w0 sample.wav)
curl -s http://127.0.0.1:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d "{
\"messages\": [
{\"role\": \"system\", \"content\": \"\"},
{\"role\": \"user\", \"content\": [
{\"type\": \"input_audio\", \"input_audio\": {\"data\": \"$AUDIO_B64\", \"format\": \"wav\"}}
]}
],
\"temperature\": 0, \"max_tokens\": 512
}"
Prompting
Like all Qwen3-ASR models, send an empty system message and an audio-only user turn
(no instruction text):
<|im_start|>system\n<|im_end|>\n
<|im_start|>user\n<|audio_start|>[audio]<|audio_end|><|im_end|>\n
<|im_start|>assistant\n
Use temperature = 0 for reproducible output.
Output format
The response is the detected language followed by the transcript. language <Name> runs
straight into the text โ there is no separator:
That is what build 9199 returns from the llama-server chat endpoint. llama-mtmd-cli on the
same build inserts an <asr_text> marker between the language and the text:
Parse both forms and do not rely on the marker:
import re
ASR_RE = re.compile(r"language\s+([A-Za-z]+)\s*(?:<asr_text>)?\s*(.*)", re.DOTALL)
def parse(raw: str) -> tuple[str | None, str]:
m = ASR_RE.search(raw.strip())
return (m.group(1), m.group(2).strip()) if m else (None, raw.strip())
The language name is ASCII and runs directly into Arabic text, so match on [A-Za-z]+ rather
than \S+ โ otherwise the first Arabic word is swallowed into the language field.
Settings that affect accuracy
These defaults are what the reference results were produced with. Changing them to save memory degrades transcription quality, usually without any error message.
| Setting | Use | Notes |
|---|---|---|
--cache-type-k / -v |
f16 |
q8_0 measurably degrades word accuracy. |
-c |
8192 |
|
-b / -ub |
2048 / 512 |
llama.cpp defaults; smaller values change results. |
--temp |
0 |
|
-n / max_tokens |
512 |
|
--jinja |
enabled | Needed for the chat template. |
Audio is resampled internally to 16 kHz mono, so any sample rate works; resampling beforehand changes nothing.
If you are memory constrained, lower -c first. It has the largest footprint (the KV cache at
-c 8192 is ~900 MB) and the least effect on quality for short clips.
Reference output
sample.wav ships with this repo: 3.11 s, 16 kHz mono WAV, DarijaCS Probe, code-switched
DarijaโFrench. The transcripts below are the 0.6B; the 1.7B runs through the same pipeline with
its own weights.
Ground truth:
Q4_K_M, build 9199, after stripping the language Darija prefix and the <asr_text> marker:
BF16, same settings:
Both 0.6B precisions return the same transcript on this clip. The model writes the French word
in Arabic script (ุงูtรฉlรฉphone โ ุงูุชูููู) and adds ููู before l'รฉcran. Q4_K_M is the
configuration used for the published evaluation.
# end-to-end check
llama-mtmd-cli -m fihria-asr-0.6b-Q4_K_M.gguf \
--mmproj mmproj-fihria-asr-0.6b-BF16.gguf \
--audio sample.wav -p " " --jinja -c 8192 -n 512 --temp 0 \
| grep '^language '
Checksums (MD5)
0.6B:
e018e4ca8b5f524c55f5409d9fe2a409 fihria-asr-0.6b-Q4_K_M.gguf
ff0e08f1bc87d7fbc5bd7cf63f08dea4 fihria-asr-0.6b-BF16.gguf
541e2927c1bbb7395adb17ce2a48e753 mmproj-fihria-asr-0.6b-BF16.gguf
1.7B:
74c956fb1c00ade7809b074362cf3ef3 fihria-asr-1.7b-Q4_K_M.gguf
e2c500f066e0675a1d4d3337dae41f47 fihria-asr-1.7b-BF16.gguf
342be56a42fa82d77be3755382c0385d mmproj-fihria-asr-1.7b-BF16.gguf
Citation
@misc{fihria2026asr,
title = {fihrIA-ASR: Open-Weight Speech Recognition and Semantic Evaluation for Moroccan Darija},
author = {{fihrIA}},
year = {2026},
url = {https://huggingface.co/fihriaLab/fihrIA-ASR-0.6B-GGUF}
}
Known limitations
- Long audio is chunked in 30 s windows by the encoder; very long files are slower and may drift.
- Fine-tuned for Darija. Other languages fall back to base Qwen3-ASR behavior and are untested here.
- Q4_K_M is the validated path: it is the configuration behind the published numbers for both sizes.
- Vocal music is transcribed as lyrics, and very noisy or instrumental clips can occasionally produce short text. For noisy production pipelines, gating with a lightweight VAD (e.g., Silero VAD) is recommended.
- Speed figures assume a modern multi-core x86-64 CPU at default thread counts.
Contact
fihrIA is a Moroccan AI research lab working on Darija speech and language. Questions about the model, the data, or a collaboration: contact@fihria.ma.
- Downloads last month
- 33
4-bit
16-bit

