leophill/w2v-bert-2.0-afrivoices-ea-6l-asr-int8-edge

Wav2Vec2-BERT 2.0 ASR for 6 East African Languages - Edge (CTranslate2 & ONNX int8)

Edge conversions of leophill/w2v-bert-2.0-afrivoices-ea-6l-asr : CTranslate2 int8 (weight mapping — CTranslate2 implements the Wav2Vec2-BERT architecture natively) and ONNX int8 (graph export with the CTC head baked in, dynamically quantised).

CTranslate2 is the recommended engine (fastest, smallest RSS); ONNX is provided for runtimes where CTranslate2 is unavailable (onnxruntime is broadly portable, incl. mobile).

One 580M-parameter model transcribes all six languages — Dholuo (luo), Kalenjin (kln), Kikuyu (kik), Maasai (mas), Somali (som), Swahili (swa) — over a shared 34-character union vocabulary (Kikuyu ĩ/ũ preserved). No language token or language input exists: the model transcribes whichever of the six languages it hears.

Built for the AfriVoices East Africa ASR Hackathon (Kaggle; Digital Umuganda & Maseno Center for Applied Artificial Intelligence) — 2nd place (provisional), macro-WER 0.34819 — and its edge constraints: <1B parameters, CPU inference within 8 GB RAM, near-real-time transcription.

⚠ CTranslate2 ≥ 4.8 output contract

Recent CTranslate2 releases (measured at 4.8.1) return encoder hidden states (T, 1024) from encode() — not CTC logits. This repository ships lm_head.npz (the 1024→34 CTC projection, fp32, ~140 KB); apply it after encode() as shown below. Older CTranslate2 builds that emit (T, 34) logits directly need no projection.

Usage — CTranslate2 (recommended, CPU, torch-free)

import re, numpy as np, soundfile as sf, ctranslate2
from transformers import SeamlessM4TFeatureExtractor, Wav2Vec2CTCTokenizer

repo = "leophill/w2v-bert-2.0-afrivoices-ea-6l-asr-int8-edge"   # this repo
model = ctranslate2.models.Wav2Vec2Bert(
    repo, device="cpu", compute_type="int8", intra_threads=4)
fe  = SeamlessM4TFeatureExtractor.from_pretrained(repo)
tok = Wav2Vec2CTCTokenizer.from_pretrained(repo)

audio_file = "audio.wav"
audio, sr = sf.read(audio_file)          # 16 kHz mono
feats = fe(audio, sampling_rate=sr, return_tensors="np").input_features
out = np.asarray(model.encode(
    ctranslate2.StorageView.from_array(feats.astype(np.float32)),
    to_cpu=True))[0]

if out.shape[-1] != 34:                   # CT2 >= 4.8: hidden states
    z = np.load("lm_head.npz")
    out = out @ z["weight"].T + z["bias"]

text = tok.decode(out.argmax(-1), group_tokens=True)   # greedy CTC
text = re.sub(" +", " ", text).strip()   # clean extra spaces
print(text)

Usage — ONNX Runtime (CPU, torch-free)

import re, numpy as np, soundfile as sf, onnxruntime as ort
from transformers import SeamlessM4TFeatureExtractor, Wav2Vec2CTCTokenizer
from huggingface_hub import hf_hub_download

repo = "leophill/w2v-bert-2.0-afrivoices-ea-6l-asr-int8-edge"      # this repo
onnx_path = hf_hub_download(repo, "onnx/model_int8.onnx")     # cached locally
sess = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
fe  = SeamlessM4TFeatureExtractor.from_pretrained(repo)
tok = Wav2Vec2CTCTokenizer.from_pretrained(repo)

audio_file = "audio.wav"
audio, sr = sf.read(audio_file)          # 16 kHz mono

enc = fe(audio, sampling_rate=sr, return_tensors="np")
mask = enc.get("attention_mask",
               np.ones(enc.input_features.shape[:2], dtype=np.int64))
logits = sess.run(["logits"], {
    "input_features": enc.input_features.astype(np.float32),
    "attention_mask": np.asarray(mask, dtype=np.int64),
})[0][0]                                   # (T, 34) — head is in the graph
# (quantisation covers MatMul/Gemm only — no ConvInteger nodes)

text = tok.decode(logits.argmax(-1), group_tokens=True)   # greedy CTC
text = re.sub(" +", " ", text).strip()   # clean extra spaces
print(text)

For best accuracy with either engine, decode with pyctcdecode + a per-language 5-gram KenLM (beam width 100; the hackathon submission used per-language α ∈ {0.25, 0.5}, β = 0.0, Swahili greedy) and window long audio at 28 s with 4 s overlap. The complete decode pipeline (windowing, stitching, LM configs) is in the solution repository.

Edge benchmark

Conditions: this checkpoint, identical clips, identical pyctcdecode + KenLM decode path, x86 CPU, Somali 200-clip long-form-skewed sample, 4 threads:

engine	ΔWER vs fp32	RTF total	peak RSS
torch fp32 (reference)	—	5.9	5.1 GB
CTranslate2 int8	−0.003	0.412	2.7 GB
ONNX Runtime int8	−0.003	0.449	4.7 GB

int8 costs no measurable WER (Δ within beam tie-breaking noise, two independent engines agreeing).

Full-test-set validation (4 threads per process — one edge device's budget): per-language RTF 0.51–0.64 (worst: Kalenjin, largest-lexicon LM); worst-case peak RSS 6.2 GB (Somali, longest clips) using sequential shard recycling to bound allocator growth — within the 8 GB envelope. Measured on a Galaxy S21: RTF 1.308 (Somali sample).

Intended use & limitations

16 kHz mono speech in the six covered languages; performance varies strongly by register (scripted read speech vs long-form spontaneous) and follows the training-data conventions of the hackathon corpora. Not a language identifier; audio outside the six languages produces unusable text.

Citation

@misc{w2v-bert-2.0-afrivoices-ea-6l-asr,
  author = {Leopold Hillah},
  title = {Jointly Finetuning Wav2Vec-BERT 2.0 for 6 East African Languages},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/leophill/w2v-bert-2.0-afrivoices-ea-6l-asr}
}
@misc{afri-voices-east-africa-asr-hackathon,
  author = {{Digital Umuganda} and {Maseno University} and
            {Maseno Center for Applied Artificial Intelligence}},
  title = {AfriVoices East Africa: ASR Hackathon},
  year = {2026},
  howpublished = {\url{https://kaggle.com/competitions/afri-voices-east-africa-asr-hackathon}},
  note = {Kaggle}
}