--- license: other license_name: openmdw-1.1 license_link: https://openmdw.ai/license/1-1/ base_model: nvidia/nemotron-3.5-asr-streaming-0.6b language: - th - en library_name: nemo pipeline_tag: automatic-speech-recognition tags: - automatic-speech-recognition - speech - streaming - Thai - FastConformer - Transducer - NeMo - Nemotron ---

OpenThai · iApp Technology

# OpenThai2.0-ASR-Streaming Streaming Thai speech-to-text with 0.64B parameters. It writes Thai speech down while people are still talking, 80 to 1120 ms behind the voice, from one model with four latency settings. It is a Thai fine-tune of NVIDIA **Nemotron 3.5 ASR Streaming 0.6B** and part of OpenThai (formerly OpenThaiGPT). Developed by iApp, launched with AIEAT. It is the model behind iApp's hosted [OpenThai ASR Realtime](https://iapp.co.th/docs/speech/speech-to-text/realtime) service. **Built on NVIDIA Nemotron.** The base model is NVIDIA's [`nvidia/nemotron-3.5-asr-streaming-0.6b`](https://huggingface.co/nvidia/nemotron-3.5-asr-streaming-0.6b), released under the OpenMDW License Agreement, version 1.1. That licence is included as [`LICENSE`](LICENSE), and NVIDIA's original model card and sub-cards are kept unchanged in [`nvidia/`](nvidia/).
ArchitectureCache-aware streaming FastConformer encoder + RNN-Transducer decoder, prompt-conditioned (NeMo EncDecRNNTBPEModelWithPrompt)
Parameters640.4 million (0.64B): encoder 609.1 million (24 FastConformer layers, 1,024 wide), language-prompt layer 4.5 million, RNN-T decoder 16.2 million (2-layer LSTM of 640) and joint network 10.7 million
Accuracy, 560 ms% of characters wrong: FLEURS 8.47, GigaSpeech2 15.35, TVSpeech 12.41, LOTUSDIS 20.77, Isan 34.90 (average 18.38; see Results)
Latency settings80, 320, 560 and 1120 ms (att_context_size [56,0], [56,3], [56,6], [56,13]), chosen at decode time
Language promptsth-TH: everything in Thai script. th-mix: English words in Latin letters
Tokenizer14,988 pieces: NVIDIA's 13,087 multilingual pieces and 1,901 new Thai pieces
Training data12,373 hours of Thai speech and 961 hours of English (LibriSpeech)
Input16 kHz mono audio (128 log-mel bands); any length when cut at pauses, as the included server does
FilesTwo .nemo files of 2.56 GB each, 32-bit weights (see Use). An ONNX export, full precision and int8, is at iapp/OpenThai2.0-ASR-Streaming-ONNX
SpeedOne NVIDIA H100: files about 257× faster than real time, about 20 live streams at once at 560 ms. A server CPU limited to 8 threads: files about 4× real time, a live stream keeps up at 560 and 320 ms. Apple M3 Max with 1 thread: files about 3× real time, a live stream keeps up at 560 and 320 ms
MemoryAbout 2.6 GB for the weights; about 6.5 GB of RAM in all on a CPU
LicenceOpenMDW-1.1
## Results ![OpenThai2.0-ASR-Streaming against Typhoon ASR Streaming 0.6B (as released and with its 4-gram LM) and the Nemotron base: average character error rate and FLEURS, GigaSpeech2, TVSpeech, LOTUSDIS and Isan at 560 ms](https://huggingface.co/datasets/iapp/OpenThai2.0-model-card-assets/resolve/main/openthai2-asr-streaming-benchmark.png) We compared it with [Typhoon ASR Streaming 0.6B](https://huggingface.co/typhoon-ai/typhoon-asr-streaming-nemotron-0.6b), the other open Thai model built on the same NVIDIA base. We decoded both ourselves, on public test sets, with the same script and settings. - At 560 ms this model makes fewer errors on read speech (FLEURS: 8.47% of characters wrong, against 11.28%), on YouTube clips (GigaSpeech2: 15.35% against 16.33%) and in meetings (LOTUSDIS: 20.77% against 28.91%, a set that favours us). - Typhoon makes fewer errors on TV talk shows (TVSpeech, its own benchmark: 10.84% against 12.41%). The two tie on the Isan dialect. - Typhoon's 4-gram language model raises its English-word recall, as its card says, but adds errors on every set here. - On whole recordings, decode in pieces cut at pauses, as the included server does. Decoded as one stream, this model drops words after quiet pauses. **560 ms** (`att_context_size` `[56,6]`), character error rate in %, lower is better: | test set | size | OpenThai2.0-ASR-Streaming | Typhoon ASR Streaming 0.6B | Typhoon + its 4-gram LM | ours − Typhoon, 95% interval | |---|---|---:|---:|---:|---| | FLEURS test, read speech | 1,021 clips | **8.47** | 11.28 | 11.73 | −2.82 [−3.15, −2.48], ours lower | | GigaSpeech2 test, YouTube | 8,823 clips, 32 recordings | **15.35** | 16.33 | 16.62 | −0.98 [−1.31, −0.58], ours lower | | TVSpeech, TV and YouTube talk | 570 clips, 18 videos | 12.41 | **10.84** | 11.05 | +1.57 [+0.90, +2.52], Typhoon lower | | LOTUSDIS meetings, conference microphone ¹ | 5,516 turns, 11 meetings | **20.77** | 28.91 | 29.02 | −8.15 [−8.88, −7.46], ours lower | | Isan dialect | 500 clips | 34.90 | 34.15 | 35.79 | +0.75 [−0.21, +1.69], tie | **1120 ms** (`att_context_size` `[56,13]`), character error rate in %, lower is better: | test set | size | OpenThai2.0-ASR-Streaming | Typhoon ASR Streaming 0.6B | Typhoon + its 4-gram LM | ours − Typhoon, 95% interval | |---|---|---:|---:|---:|---| | FLEURS test, read speech | 1,021 clips | **8.36** | 11.32 | 11.64 | −2.96 [−3.31, −2.65], ours lower | | GigaSpeech2 test, YouTube | 8,823 clips, 32 recordings | **15.23** | 16.18 | 16.44 | −0.95 [−1.35, −0.49], ours lower | | TVSpeech, TV and YouTube talk | 570 clips, 18 videos | 12.14 | **10.70** | 11.18 | +1.45 [+0.77, +2.46], Typhoon lower | | LOTUSDIS meetings, conference microphone ¹ | 5,516 turns, 11 meetings | **20.30** | 28.23 | 28.64 | −7.93 [−8.66, −7.24], ours lower | | Isan dialect | 500 clips | 34.49 | 33.73 | 35.68 | +0.76 [−0.28, +1.86], tie | Bold: lower than the other model in that row with 95% confidence (paired bootstrap, see below). **Whole recordings**, 560 ms. Each file joins every clip of one video or recording, in order, with one second of silence between clips. *One stream* decodes the file in one pass, as NeMo's script does; *cut at pauses* decodes it the way [`serve/`](serve/) does, in pieces cut where nobody speaks for 0.7 s, each decoded on its own (both models through the same code, Typhoon at its released settings). | test set | size | one stream: ours | one stream: Typhoon | ours − Typhoon, 95% interval | cut at pauses: ours | cut at pauses: Typhoon | ours − Typhoon, 95% interval | |---|---|---:|---:|---|---:|---:|---| | TVSpeech, whole videos (0.5–47 min) | 18 files | 23.54 | **11.07** | +12.47 [+7.49, +17.08], Typhoon lower | 12.66 | **10.84** | +1.82 [+1.16, +2.74], Typhoon lower | | GigaSpeech2 test, whole recordings (5–87 min) ² | 31 files | **23.10** | 30.07 | −6.97 [−12.06, −2.63], ours lower | **13.53** | 14.42 | −0.88 [−1.34, −0.41], ours lower | Decoded as one stream, this model loses words after a pause of near-silence: the longer and quieter the pause, the more it drops (see [Limits](#limits)). Cut at pauses, it comes back near its clip-by-clip error on TVSpeech (12.66 whole against 12.41 clip by clip) and below it on GigaSpeech2 (13.53 against 15.35). **By latency**, character error rate in %, ours / Typhoon (as released): | test set | 80 ms | 320 ms | 560 ms | 1120 ms | |---|---:|---:|---:|---:| | FLEURS test, read speech | 8.73 / 11.78 | 8.53 / 11.67 | 8.47 / 11.28 | 8.36 / 11.32 | | GigaSpeech2 test, YouTube | 16.64 / 17.75 | 15.53 / 16.64 | 15.35 / 16.33 | 15.23 / 16.18 | | TVSpeech, TV and YouTube talk | 13.06 / 11.44 | 12.54 / 10.84 | 12.41 / 10.84 | 12.14 / 10.70 | | LOTUSDIS meetings, conference microphone | 26.12 / 34.35 | 21.79 / 29.49 | 20.77 / 28.91 | 20.30 / 28.23 | | Isan dialect | 35.84 / 35.37 | 34.72 / 34.65 | 34.90 / 34.15 | 34.49 / 33.73 | **Typhoon with a tuned blank offset.** Our released file stores a blank offset of −1.0, chosen on a development set. Typhoon's best offset on the same development set is −2.0. With it, at 560 ms: | test set | ours | Typhoon, offset −2.0 | ours − Typhoon, 95% interval | |---|---:|---:|---| | FLEURS test, read speech | 8.47 | 11.63 | −3.17 [−3.50, −2.83], ours lower | | GigaSpeech2 test, YouTube | 15.35 | 15.26 | +0.10 [−0.38, +0.63], tie | | TVSpeech, TV and YouTube talk | 12.41 | 10.68 | +1.73 [+1.14, +2.61], Typhoon lower | | LOTUSDIS meetings, conference microphone | 20.77 | 26.71 | −5.94 [−6.50, −5.41], ours lower | | Isan dialect | 34.90 | 33.87 | +1.03 [+0.24, +1.91], Typhoon lower | **English words** on TVSpeech at 560 ms: the share of the reference's English words written in Latin letters (recall), the share of written English words that are right (precision), and their F1; exact match, ignoring case. | | recall | precision | F1 | |---|---:|---:|---:| | ours, `th-mix` | 13.7 | 22.6 | 17.0 | | ours, `th-TH` | 0.5 | 75.0 | 0.9 | | Typhoon | 16.1 | 30.6 | 21.1 | | Typhoon + 4-gram | 18.9 | 29.0 | 22.9 | **By the method of the public Thai ASR Leaderboard** (pilot 2026.10: spoken-form normalisation, six weighted groups), 1120 ms. This is our own measurement, not an entry on that board. | | FLEURS | Isan | TVSpeech | GigaSpeech2 | close mics | far mics | weighted | |---|---:|---:|---:|---:|---:|---:|---:| | OpenThai2.0-ASR-Streaming | 8.29 | 33.85 | 11.90 | 15.08 | 22.44 | 54.99 | 22.15 | | Typhoon ASR Streaming 0.6B | 11.20 | 33.30 | 10.59 | 16.18 | 30.72 | 72.11 | 26.25 | | Typhoon + 4-gram | 11.54 | 35.26 | 11.07 | 16.44 | 31.07 | 73.05 | 26.91 | How we measured: - Every model was decoded by us with NeMo's cache-aware streaming script (`speech_to_text_cache_aware_streaming_infer.py`), greedy, `max_symbols_per_step=2`, prompt `th-TH` (with NeMo's default of 10 instead, Typhoon's error moves by at most 0.06 points on FLEURS, TVSpeech and GigaSpeech2). Typhoon is [`typhoon-ai/typhoon-asr-streaming-nemotron-0.6b`](https://huggingface.co/typhoon-ai/typhoon-asr-streaming-nemotron-0.6b) as released, and with its paired 4-gram ([`...-0.6b-ngram`](https://huggingface.co/typhoon-ai/typhoon-asr-streaming-nemotron-0.6b-ngram), weight 0.5, as its card says). - Both sides are normalised the same way before scoring: Thai digits and spoken numbers made equal, spaces removed; then true edit distance over all characters of a set. - Intervals are 95% paired bootstrap intervals that resample whole videos (TVSpeech), recordings (GigaSpeech2), speakers (Isan), meetings (LOTUSDIS) or files (joined sets). An interval that includes zero is a tie. - Latency is the audio chunk the model waits for, (right context + 1) × 80 ms. Typhoon's card counts only the look-ahead, so its "1040 ms" is the 1120 ms setting here. - ¹ We trained on other LOTUSDIS meetings from the same rooms and microphones, and some speakers appear in both, so that row favours us. TVSpeech is Typhoon's own benchmark. We trained on FLEURS's training and development splits, not its test split. Both models trained on GigaSpeech2's training split. - ² GigaSpeech2 test recording 26 is left out of the joined set: its transcripts do not match its audio (about 70% errors for every model). It stays in the 8,823-clip set above, as published. ## Use There are two files. Use the first unless you run the reference server. | file | for | |---|---| | `openthai2.0-asr-streaming_bias-1.0.nemo` | NeMo's streaming script and your own code. The blank offset −1.0 that every number above uses is stored in the file. | | `openthai2.0-asr-streaming_raw.nemo` | The reference server in [`serve/`](serve/), which sets the blank offset per mode itself. | Both need NVIDIA NeMo (Speech) with cache-aware streaming and `EncDecRNNTBPEModelWithPrompt`. We tested NeMo Speech at commit `cf724ac`, PyTorch 2.13 and Python 3.12: ```bash uv venv asr --python 3.12 && source asr/bin/activate uv pip install torch==2.13.0 --index-url https://download.pytorch.org/whl/cu130 # or the CPU wheel uv pip install Cython packaging uv pip install --extra-index-url https://download.pytorch.org/whl/cu130 --index-strategy unsafe-best-match \ "nemo-toolkit[asr] @ git+https://github.com/NVIDIA-NeMo/Speech.git@cf724ac337d1ebc7d0dda1e23fb80916f52927a5" hf download iapp/OpenThai2.0-ASR-Streaming --local-dir OpenThai2.0-ASR-Streaming ``` ### Transcribe files with NeMo's streaming script This is exactly how every benchmark number on this page was produced. Audio must be 16 kHz mono. ```bash echo '{"audio_filepath": "a.wav", "duration": 8.8, "text": ""}' > in.jsonl # one line per file python /examples/asr/asr_cache_aware_streaming/speech_to_text_cache_aware_streaming_infer.py \ model_path=OpenThai2.0-ASR-Streaming/openthai2.0-asr-streaming_bias-1.0.nemo \ dataset_manifest=in.jsonl output_path=out batch_size=32 \ att_context_size="[56,6]" target_lang=th-TH strip_lang_tags=true \ rnnt_decoding.greedy.max_symbols_per_step=2 # result: out/streaming_out_*.json, one line per file, field "pred_text" ``` - `att_context_size`: `[56,0]` 80 ms, `[56,3]` 320 ms, `[56,6]` 560 ms, `[56,13]` 1120 ms. - `target_lang`: `th-TH`, or `th-mix` to write English words in Latin letters. - `max_symbols_per_step=2` stops rare repetition loops at no cost in accuracy. - **The script decodes each file as one stream.** Cut long recordings at pauses first, or use the reference server below, which does it for you. In one stream, a pause of near-silence makes the model drop some of the speech that follows (see [Limits](#limits)). - Use `dataset_manifest=`; with `audio_file=` this NeMo commit prints the text and then stops with an error. - Do not use NeMo's `model.transcribe()` for this model: at this commit its data loader ignores `target_lang`, so it stops with `Unknown prompt key: 'None'`, and its default prompt mode would use the language-agnostic prompt for about half of the clips. ### Reference server: files and live streams [`serve/`](serve/) is a small FastAPI service around the model: `POST /v1/transcribe` for files and `WS /v1/live` for live audio, with three modes and optional speaker labels. It runs on a GPU or a CPU. Both cut long audio at pauses, the way iApp's hosted service does. A file longer than 30 seconds is cut where nobody speaks for 0.7 s and each piece is decoded on its own; the response lists them in `segments`. The exception is a file with speaker labels, which is still decoded as one stream. A live stream sends a final text each time the speaker pauses. ```bash uv pip install fastapi uvicorn python-multipart websockets cd OpenThai2.0-ASR-Streaming/serve CHINDA_DEVICE=cuda CHINDA_PRELOAD=1 python -m uvicorn app:app --port 18081 # CPU: leave out CHINDA_DEVICE curl -F file=@a.wav -F mode=standard http://127.0.0.1:18081/v1/transcribe ``` | `mode` | for | what it does | |---|---|---| | `standard` (default) | read speech, news, lectures, dictation | blank offset −1.0; clips under 2 s decoded again at −2.5; empty output decoded again at −4.0 | | `conversation` | YouTube, social media, casual talk | blank offset −2.0; empty output decoded again at −4.0 | | `meeting` | several people in a room | blank offset −1.0, and speaker labels from [`nvidia/Nemotron-3-Diarization`](https://huggingface.co/nvidia/Nemotron-3-Diarization) (downloaded on first use) | Other fields: `lang` (`th-TH` or `th-mix`) and `latency_ms` (80, 320, 560 or 1120). Each mode's settings were chosen on development data and then measured once on test data. Live stream over WebSocket (audio as 16 kHz mono 16-bit PCM): ```python import json, soundfile as sf from websockets.sync.client import connect x = sf.read("a.wav", dtype="float32")[0] # 16 kHz mono pcm = (x.clip(-1, 1) * 32767).astype("