oluwagbotty/nemo-ctc-lr1e-4

NeMo Conformer-CTC for Kinyarwanda speech, fine-tuned on stuttered speech

A small (about 120 million parameters, 487 MB) Kinyarwanda speech recognition model. It is nvidia/stt_rw_conformer_ctc_large (NVIDIA NeMo Conformer-CTC, trained on about 2,000 hours of Common Voice Kinyarwanda) fine-tuned on the CDLI non-standard speech dataset, whose speakers are mostly young adults who stutter.

It is a CTC (connectionist temporal classification) model: it decodes in one pass, so it cannot get stuck in repetition loops, and it runs fast on a CPU. It writes lower-case letters, spaces and apostrophes only, with no punctuation.

It was made for a research project on low-cost Kinyarwanda speech recognition for people who stutter (Rwanda Innovation Sprint, run by CDLI).

How to use

import nemo.collections.asr as nemo_asr
from huggingface_hub import hf_hub_download

path = hf_hub_download("oluwagbotty/nemo-ctc-lr1e-4", "best.nemo")
model = nemo_asr.models.EncDecCTCModelBPE.restore_from(path, map_location="cpu")
out = model.transcribe(["clip_16khz_mono.wav"])
print(out[0].text if hasattr(out[0], "text") else out[0])  # newer NeMo versions return objects with a .text field

The audio must be 16 kHz, mono. Install with pip install "nemo_toolkit[asr]".

Results

Test split of the CDLI Kinyarwanda dataset, 1,169 recordings from 9 speakers, greedy decoding. We score the way the CDLI model cards do: apostrophes are removed and the words joined, then Whisper's basic text normaliser is applied. Pooled WER counts all words together. Per-utterance WER is the average of the WER of each recording (capped at 100%).

Model	Parameters	Pooled WER (%)	Per-utterance WER (%)	CER (%)	Loops
This model	0.12B	39.7	41.0	13.0	0
w2v-BERT 2.0, same fine-tuning data (our run)	0.6B	38.7	40.6	11.7	0
CDLI Whisper large-v3 (loops fixed)	1.55B	39.9	38.4	13.7	17 fixed
CDLI Whisper small (loops fixed)	0.24B	47.6	45.2	17.5	17 fixed
This model before fine-tuning (50 clips only)	0.12B	59.6	n/a	23.1	0

• Across the 9 test speakers, this model minus w2v-BERT is +1.0 pooled WER points (95% interval -0.8 to +3.3) and minus Whisper large is -0.2 (-3.7 to +4.7). So it is on par with both. It is clearly better than Whisper small (-7.9, -11.9 to -2.5). We did not compute intervals for CER or per-utterance WER.
• Text prompts (read sentences): pooled WER 36.2. Picture prompts (free descriptions): 43.6. With plain scoring (no apostrophe step): WER 42.4, CER 13.5.
• Per-utterance WER by severity group, averaged per speaker (3 speakers each, do not over-read): mild 37.8, moderate 38.9, severe 46.0.
• Validation split (6 speakers, used to choose the checkpoint): pooled WER 44.6.
• A second run with a learning rate of 5e-5 gave pooled WER 40.7 on the same test split, so the learning rate was not the limit.

Speed. On a GPU: 0.05 s per recording. On a server CPU (molab, 10 recordings of 15 s on average, fp32): 0.28 s per recording with 10 threads and 0.90 s with 2 threads (real-time factor 0.022 and 0.065). This was not measured on a typical laptop or phone.

Training

• Starting point: nvidia/stt_rw_conformer_ctc_large. Train split only. Dev split for choosing the checkpoint.
• AdamW, learning rate 1e-4, warm-up 10% then cosine decay, weight decay 1e-3, batch size 16, 6 epochs (4,326 steps), mixed precision, one GPU, about 22 minutes. The best checkpoint on dev (validation WER 0.4568, against 0.6715 before fine-tuning) was kept.
• The batch-norm statistics of the pretrained model were kept fixed. With small batches they were overwritten and the validation WER got much worse.
• Targets were lower-cased and reduced to the model's characters. Stutter repetitions were kept.

Data and attribution

Data sourced from the CDLI Non-Standard Speech Dataset, Centre for Digital Language Inclusion, UCL, 2025. This project was funded by UK International Development through the AT2030 Programme with support from Google.org.

The model was fine-tuned on the train split of the CDLI Rwandan Kinyarwanda non-standard speech dataset v1.0 (access is gated; accept the CDLI terms and conditions there). The dataset is not included in this repository and cannot be shared from it.

• Train: 11,536 recordings from 46 speakers (47.7 hours). The speakers are mostly young adults. Most of them stutter, and a few have other speech differences.
• The dev split (6 speakers) was used only to choose the checkpoint. The test split (9 speakers) was scored once, after training.
• The extra split was never used.
• No speaker is in both train and dev or test.
• The audio was recorded in Rwanda in 2026, indoors with low noise, on a tablet app, from picture prompts (free descriptions) and text prompts (sentences that the speaker reads).

Limitations and risks

• Few test speakers. The test split has 9 speakers (3 mild, 3 moderate, 3 severe). A difference of less than about 3 WER points between two models cannot be confirmed with this many speakers. Each speaker group in the tables has only 3 speakers, so do not read conclusions from them.
• Verbatim output. The training transcripts are verbatim, so the model often writes repetitions as they were said (for example "mu mu mu mwaka"). Blocks and prolongations are not visible in text. If you need the fluent sentence, clean the text after decoding.
• Spelling and spacing. The transcripts use competing spelling and word-spacing conventions (for example "uyumunsi" and "uyu munsi"). Mapping spelling and spacing variants learned from the train transcripts to one form lowers WER by only about 2.5 points (6 to 7% of the errors); most errors are real model errors, many one or two letters off. Report CER next to WER.
• Speakers and conditions. The data is mostly young adults (18 to 30), mostly male (44 of the 61 speakers in the full dataset), recorded indoors with low noise. Performance on children, older speakers, other dialects, noisy places or phone audio is not known. Whispered speech is not covered.
• Long clips. About a quarter of the recordings stop at about 25 seconds because of an app limit, so speech may be cut off in the data.
• Prompt type. Sentences that speakers read in the text prompts also appear in the train transcripts, so text-prompt results are easier than free speech. Picture-prompt results are the better guide for open speech.
• Not for decisions about people. This model must not be used to diagnose or assess any speech condition, or for legal, law enforcement or clinical decisions. It must not be used to identify speakers or to profile or monitor people. The CDLI terms and conditions forbid re-identifying speakers, building voices that mimic a speaker, and training models that promote exclusion or harm.
• Errors can hurt. A wrong transcript can change the meaning of what a person said. Show the text to the speaker so that they can check it, and do not act on it automatically.

Citation

If you use this model, please also cite the dataset:

@misc{cdli_speech_2026,
  author = {Centre for Digital Language Inclusion (CDLI), a project of UCL's (University College London's) GDI Hub (Global Disability Innovation Hub)},
  title = {CDLI Non-Standard Speech Datasets},
  year = {2026},
  url = {https://huggingface.co/cdli/datasets}
}