LuoYiSULIXAY/fastconformer-streaming-lao

FastConformer Streaming Lao

A real-time, streaming Lao (ພາສາລາວ) speech recognition model. It writes words while the speaker is still talking: the audio is processed in chunks, and the model never waits for the end of the sentence.

It is NVIDIA's cache-aware streaming FastConformer (stt_en_fastconformer_hybrid_large_streaming_multi, 114M parameters), fine-tuned on Lao with a new Lao tokenizer. It has two decoders, RNN-T (more accurate, recommended) and CTC. Four look-ahead sizes (0, 80, 480 and 1040 ms) are built into the same checkpoint.

Credits

• Developed by: VILAIPHONE Sulixay, Savath Saypadith, Makmeephon Xaysana, SOUNAKHEN Phonesavanh — CEIT
• In collaboration with / supported by: Laligence
• Reviewers: 3 Lao native-speaker annotators
• Base model: FastConformer Hybrid Large Streaming Multi by NVIDIA (CC-BY-4.0)

Results at a glance

Accuracy, meeting2 (1,529 clips)	CER 10.62 (RNN-T, 480 ms look-ahead)
Accuracy, test_blind (247 clips)	CER 3.30
Words appear	about 0.6 s after they are spoken (560 ms chunks)
Speed, one live stream, GPU A800	28 ms of compute per 560 ms chunk (RTF 0.052)
Speed, one live stream, CPU (8 threads)	53 ms of compute per 560 ms chunk (RTF 0.099)
Size	114M parameters, 459 MB .nemo file

Files

File	What it is
fastconformer-streaming-lao.nemo	the model (NeMo checkpoint, includes the Lao tokenizer)
streaming.py	a small helper for live (chunk-by-chunk) decoding, with a demo that streams a WAV file

Usage

Install NVIDIA NeMo (pip install "nemo_toolkit[asr]"; tested with NeMo 3.0).

Whole files (offline)

from huggingface_hub import hf_hub_download
from nemo.collections.asr.models import ASRModel

path = hf_hub_download("LuoYiSULIXAY/fastconformer-streaming-lao", "fastconformer-streaming-lao.nemo")
model = ASRModel.restore_from(path).eval()
model.change_decoding_strategy(decoder_type="rnnt")      # or "ctc"
model.encoder.set_default_att_context_size([70, 6])      # look-ahead: [70,0]=0 ms, [70,1]=80 ms, [70,6]=480 ms, [70,13]=1040 ms
print(model.transcribe(["example.wav"])[0].text)         # 16 kHz mono

Decoding a whole clip with a given look-ahead gives the same text as streaming it at that look-ahead (in our check, 29 of 30 clips were identical; the other one ended in the middle of a word, with no audio after it).

Live streaming

from streaming import Streamer   # streaming.py from this repository

model.change_decoding_strategy(decoder_type="rnnt")
model.encoder.set_default_att_context_size([70, 6])
s = Streamer(model)
for samples in microphone_chunks():          # float32, 16 kHz mono, any chunk size (e.g. 100 ms)
    s.push(samples)
    if s.advance():                          # runs every complete 560 ms chunk; True if the text changed
        print(s.text)
s.advance(final=True)                        # at a pause or the end: flush the last chunk
print(s.text)
s.reset()                                    # start the next utterance

Demo with a file: python streaming.py fastconformer-streaming-lao.nemo example.wav 480.

For long live sessions, cut the stream at pauses (for example with Silero VAD) and call advance(final=True) then reset() at each pause. That is how we serve it, and it keeps each utterance under about 15 seconds.

Evaluation

CER is computed without spaces, after the reference and the output are both normalized to the spelling rules below. Syll-WER is the word error rate over Lao syllables. All numbers use the same clips and references.

This model, by look-ahead and decoder. Look-ahead is how much future audio the model sees before it writes; the chunk size is the look-ahead plus 80 ms.

Look-ahead	Chunk	RNN-T meeting2	RNN-T test_blind	CTC meeting2	CTC test_blind
0 ms	80 ms	11.51	3.64	12.23	4.00
80 ms	160 ms	11.15	3.30	11.86	3.78
480 ms	560 ms	10.62	3.30	11.06	3.61
1040 ms	1120 ms	10.40	3.10	10.75	3.41

Syll-WER for RNN-T at 480 ms: 14.58 on meeting2 and 4.99 on test_blind. RNN-T is better than CTC at every look-ahead. 480 ms is the setting we recommend: it is nearly as accurate as 1040 ms, with half the delay.

Compared with our offline (non-streaming) Lao models, trained on the same data. These see the whole clip, up to 30 s, before writing anything.

Model	Parameters	Streaming	meeting2	test_blind
This model (RNN-T, 480 ms)	114M	yes	10.62	3.30
Whisper large-v3-turbo, fine-tuned	809M	no	9.47	3.01
Qwen3-ASR 1.7B, fine-tuned (Run I)	1.7B	no	8.44	2.41

On meeting2 this model is 2.18 CER points behind the Qwen3-ASR 1.7B model (95% paired bootstrap CI 1.87–2.50), and about 1.2 points behind Whisper turbo. In return it is 7–15 times smaller, streams, and runs on a CPU. It almost never gets stuck repeating a word: 1 runaway output in 1,776 test clips.

Test set	Clips	Domain
meeting2	1,529	formal Lao meetings (3 videos), 168.5 min
test_blind	247	mixed; transcribed by a reviewer who saw no model output

Speed (real-time factor)

RTF is compute time divided by audio duration: 0.05 means 1 minute of audio takes 3 seconds. All runs use RNN-T at 480 ms look-ahead, on the meeting2 audio (CPU: the first 200 clips, 23.1 min).

Hardware	Mode	RTF	Faster than real time
GPU NVIDIA A800 80GB	offline, batch 32	0.0006	1,649×
GPU NVIDIA A800 80GB	offline, batch 1	0.0055	181×
GPU NVIDIA A800 80GB	streaming, one live stream	0.052	19×
CPU Xeon Platinum 8350C, 8 threads	offline, batch 1	0.018	55×
CPU Xeon Platinum 8350C, 8 threads	streaming, one live stream	0.099	10×

In streaming mode each 560 ms chunk takes 28 ms on the GPU (95th percentile 28.4 ms), or 53 ms on the CPU (95th percentile 56 ms). That is well inside the 560 ms budget, so a live stream never falls behind, even on a CPU. Streaming RTF is higher than offline because the encoder runs once per chunk with batch 1. The helper also recomputes the input features over the current utterance each step, which is simple but not optimal.

In a live test through our web demo, the first words appeared about 1.0 s after speech started, and the text then grew every 0.5–0.6 s. Each sentence was final about 0.1–0.2 s after the pause was detected.

Training setup.

• Started from the English streaming checkpoint. A new Lao SentencePiece BPE tokenizer (1,024 tokens) replaced the English one, so the RNN-T and CTC output layers were learned from scratch.
• All four look-ahead sizes were trained together. SpecAugment as in the base recipe.
• AdamW with a cosine schedule: peak learning rate 3e-4, 2,000 warm-up steps. Batch 96 on 3 GPUs for steps 0–14,000, then batch 192 on 6 GPUs (the learning-rate schedule was rescaled at the switch).
• Validation CER was checked every 1,000–2,000 steps, with early stopping after 5 checks without improvement.
• The best checkpoint was at step 18,000 (about 28 epochs, validation CER 5.53). Training stopped at step 23,000 and took about 3.5 hours on NVIDIA A800 GPUs.

Spelling rules

Outputs and references follow one Lao spelling policy:

• ຳ, not ໍາ.
• ໜ, ໝ, ຫຼ, not ຫນ, ຫມ, ຫລ, except in abbreviations such as ທຫລ.
• Repeated marks are collapsed, and ໆ is written out.

Text that uses other conventions will show extra "errors" that are only spelling differences. The raw output keeps some spaces between words, as in the training labels; the scores above ignore spaces.

Limitations

• Less accurate than large offline models. Use an offline model when you do not need live text and accuracy matters most.
• Clip endings. If the audio stops in the middle of a word, with no pause after it, the last word can be lost in streaming mode. Live speech normally ends with a pause, so this rarely matters in practice.
• Domain. Trained mostly on read speech and formal meetings. Expect higher error on casual conversation, strong regional accents, overlapping speakers, music or noise.
• Hardest content. Lao mixed with Thai or English words, personal and place names, and numbers cause the most errors.
• Possible label bias. Part of the meeting2 references were made by reviewers who edited a model's draft. Such labels can lean toward the model that produced the draft, so treat small differences between models on meeting2 with care.

Intended use

Live captions, voice input and real-time transcription of Lao speech; drafting transcripts for research, accessibility and data collection, with human review where accuracy matters. Do not use it to identify or monitor people, or as the only record in legal, medical or other high-stakes settings.

License

CC-BY-4.0, the same as the base model. When you use or redistribute it, credit this model and NVIDIA's FastConformer.

Acknowledgements

We thank CEIT and Laligence for compute, data collection and review support, the Lao reviewers who checked the labels, and NVIDIA for releasing the streaming FastConformer and NeMo.

Citation

A paper describing the data pipeline is in preparation; this section will be updated with its citation.

@misc{fastconformer_streaming_lao,
  title  = {FastConformer Streaming Lao: a real-time Lao speech recognition model},
  author = {VILAIPHONE, Sulixay and Saypadith, Savath and Xaysana, Makmeephon and SOUNAKHEN, Phonesavanh},
  note   = {CEIT, in collaboration with Laligence},
  year   = {2026},
  url    = {https://huggingface.co/LuoYiSULIXAY/fastconformer-streaming-lao}
}