Flo976/w2v-bert-malagasy

w2v-BERT 2.0 — Malagasy speech recognition, robust on short utterances

13.4 % word error rate on short Malagasy utterances (2–8 s), against 21.5 % for a Whisper medium fine-tuned on the same data — and 5× faster on GPU.

This CTC model transcribes Malagasy speech. It is built for voice assistants and dictation, where speech arrives as short segments cut by voice-activity detection. Most Malagasy ASR models are trained and measured on 15–30 s recordings, and make about 65 % more errors on short segments; this one was trained on both.

Try it: Flo976/malagasy-asr-demo · Language model: Flo976/malagasy-kenlm-4gram

Résumé en français. Modèle de reconnaissance vocale malgache (w2v-BERT 2.0, CTC), entraîné pour rester fiable sur les énoncés courts d'un assistant vocal. Sur 11 625 énoncés de 2 à 8 s jamais vus à l'entraînement, il fait 13,4 % d'erreurs de mots avec le modèle de langue, contre 21,5 % pour un Whisper medium fine-tuné sur les mêmes données, et répond 5 fois plus vite. Il ne régresse pas sur les extraits longs (10,6 %). Sortie en minuscules, sans ponctuation.

Results

Short utterances (2–8 s) — the voice-assistant regime

11,625 utterances (16.0 h, 94 speakers), cut from the test split by forced alignment at word boundaries. No test speaker appears in training.

Model	Decoding	WER	CER
this model	KenLM 4-gram + apostrophe rule	13.40 %	3.74 %
this model	KenLM 4-gram	13.63 %	3.76 %
this model	greedy	14.33 %	3.76 %
badrex/w2v-bert-2.0-malagasy-asr	KenLM 4-gram	17.77 %	5.45 %
badrex/w2v-bert-2.0-malagasy-asr	greedy	19.87 %	5.79 %
Flo976/whisper-malagasy-medium	greedy	21.49 %	8.97 %

Each comparison quoted on this card is significant at 95 % (paired bootstrap on the same utterances, 2,000 resamples). Against the Whisper medium, this model gains 8.1 points (95 % CI [−8.64, −7.64]). At equal decoding this model beats its starting point badrex by 5.5 points greedy (95 % CI [−5.76, −5.34]) and by 4.1 points with the language model ([−4.34, −3.93]).

Full-length clips — no regression

The 3,101 original test clips (16.5 h, median 18 s).

Model	Decoding	WER	CER
this model	KenLM 4-gram	10.60 %	2.65 %
badrex/w2v-bert-2.0-malagasy-asr	KenLM 4-gram	10.86 %	2.67 %
this model	greedy	12.01 %	2.85 %
badrex/w2v-bert-2.0-malagasy-asr	greedy	12.20 %	2.88 %
Flo976/whisper-malagasy-medium	greedy	13.07 %	3.48 %

Speed

One utterance at a time (2–8 s), as a voice assistant receives them:

Model	Hardware	Median latency	90th percentile
this model, greedy	RTX 5070 Ti	52 ms	65 ms
this model, with KenLM	RTX 5070 Ti	74 ms	92 ms
Whisper medium (Flo976)	RTX 5070 Ti	259 ms	648 ms
this model, greedy	2 CPU cores (Intel Core Ultra 7 265K)	1.45 s	1.53 s

A CTC model reads the audio in one pass; Whisper writes its output token by token, so its latency grows with the sentence. On CPU the acoustic model dominates: the language model adds no measurable time. Set PyTorch's thread count to the cores actually allotted (torch.set_num_threads): in a container, the default host count oversubscribes the CPU and triples latency.

Usage

Output is lowercase, without punctuation, with numbers written as digits. Audio must be 16 kHz mono.

Greedy decoding

import librosa
import torch
from transformers import AutoModelForCTC, AutoProcessor

processor = AutoProcessor.from_pretrained("Flo976/w2v-bert-malagasy")
model = AutoModelForCTC.from_pretrained("Flo976/w2v-bert-malagasy").eval()

audio, _ = librosa.load("audio.wav", sr=16000, mono=True)
inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
print(processor.batch_decode(logits.argmax(-1))[0])

With the Malagasy language model (recommended)

pip install pyctcdecode https://github.com/kpu/kenlm/archive/master.zip

import re
from huggingface_hub import hf_hub_download
from pyctcdecode import build_ctcdecoder

tok = processor.tokenizer
vocab = tok.get_vocab()
labels = ["" if t == tok.pad_token else " " if t == tok.word_delimiter_token else "⁇" if t == tok.unk_token else t
          for t in sorted(vocab, key=vocab.get)]
lm_repo = "Flo976/malagasy-kenlm-4gram"
unigrams = [w.strip() for w in open(hf_hub_download(lm_repo, "unigrams.txt"), encoding="utf-8") if w.strip()]
decoder = build_ctcdecoder(labels, kenlm_model_path=hf_hub_download(lm_repo, "mg_4gram.bin"),
                           unigrams=unigrams, alpha=0.3, beta=-1.0)  # beta=0.0 for clips over 10 s

text = decoder.decode(logits[0].numpy(), beam_width=100)
text = re.sub(r"'\s+", "'", re.sub(r"\s+'", "'", text))  # "amin' ny" -> "amin'ny"
print(text)

The parameters were tuned on the validation split: α = 0.3 with β = −1.0 for utterances of 10 s or less, β = 0.0 for longer clips. A spell-checking pass on top of the language model was measured and hurts (+0.17 point); it helps only without the language model (−0.38 point).

Recordings longer than about 30 s are best split at pauses; see app.py in the demo for a simple approach.

Training

Starting point	badrex/w2v-bert-2.0-malagasy-asr, itself fine-tuned from facebook/w2v-bert-2.0
Data	train split of badrex/malagasy-speech-full: 28,168 full clips (147.7 h) + 105,284 short segments (144.0 h) cut from those same clips
Short segments	CTC forced alignment of the reference transcript, cut at word boundaries into 2–8 s pieces
Objective	CTC, character vocabulary (a–z, 0–9, apostrophe, hyphen)
Schedule	learning rate 1e-5, cosine, 500 warm-up steps; up to 400 s of audio per optimizer step
Regularization	SpecAugment, time mask 0.05, feature mask 0.10
Precision	bfloat16, gradient checkpointing
Length	best checkpoint at step 3,200 (0.65 epoch); stopped at step 4,800 after four evaluations without improvement
Hardware	one RTX 5070 Ti (16 GB), about 3.5 hours

The gain comes from the short segments: continuing badrex's training on full-length clips alone, with the same recipe, gave no improvement on validation over 0.75 epoch.

Evaluation protocol

• Speaker-disjoint splits: 56 speakers in train, 95 in validation, 94 in test, no overlap (verified).
• Shared normalization of references and hypotheses: Unicode NFC, lowercase, punctuation removed, apostrophes and hyphens kept inside words.
• Choices on validation only: checkpoint selection, language-model parameters (tuned on half the speakers, checked on the other half) and post-processing rules. The test split was used once per frozen configuration.
• Significance: paired bootstrap on the same utterances; a difference is reported only when its 95 % confidence interval excludes zero.

Limitations

• Short utterances are a proxy. They are cut from read descriptions of images, not conversational speech captured by a microphone and a voice-activity detector. Results on real assistant traffic may differ.
• Segment boundaries come from badrex's alignment, which may slightly favour models of the same family over Whisper on the short-utterance set.
• Under 3 s remains the weakest regime: 15.9 % WER with the language model.
• Single training run: run-to-run variance was not measured.
• Domain and dialect: the training data is read speech, mostly official Malagasy (Merina). Coastal dialects and noisy environments are not evaluated.
• No punctuation or casing, numbers as digits.

License and attribution

This model is released under CC-BY-4.0. It is a derivative of badrex/w2v-bert-2.0-malagasy-asr by Badr al-Absi (CC-BY-4.0), itself derived from facebook/w2v-bert-2.0 (MIT). Please credit both when you reuse it.

The training data, badrex/malagasy-speech-full, does not declare a license or a source on the Hub. The language model is published separately under CC-BY-SA-3.0, because it is built from Malagasy Wikipedia.

Citation

@misc{w2v-bert-malagasy-2026,
  author    = {Florent Didelot},
  title     = {w2v-BERT 2.0 for Malagasy speech recognition, robust on short utterances},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/Flo976/w2v-bert-malagasy}
}

Developed for Milo Voice, a Malagasy voice assistant, by Flo976 — Florent Didelot / Sooatek. Questions and feedback, especially from native speakers, are welcome in the discussions.