litert-community/Breeze-ASR-26-LiteRT

Breeze-ASR-26 — LiteRT

Breeze-ASR-26 (BreezeASR-Taigi) transcribes Taiwanese Hokkien (Taigi) speech into Mandarin Chinese characters. It is a 1.55B-parameter Whisper large-v2 fine-tune by MediaTek Research and NYCU (paper).

This repository packages it as one int8 .tflite file for CPU inference with LiteRT. Unlike the other Whisper exports in this organization, the decoder keeps a key/value cache, so each step runs the decoder on one new token instead of the whole token window. Tested on an x86 desktop and a Samsung Galaxy A71 5G.

Files

File	Size	Description
breeze_asr_26_30s_i8_kv256.tflite	1.57 GB	Dynamic-range int8 weights, float32 activations; 30 s window; 256-position decoder cache
transcribe.py	7 KB	Reference loop in Python: features, greedy decoding, 30 s windows

Signatures

Signature	Inputs	Outputs
encode	input_features float32 [1, 80, 3000] (log-mel, one 30 s window)	cross_kv_k_{i}, cross_kv_v_{i} float32 [1, 1500, 20, 64], i = 0…31
decode	tokens int32 [1, 1], input_pos int32 [1], kv_cache_k_{i}, kv_cache_v_{i} float32 [1, 256, 20, 64], and the cross_kv_* tensors	logits float32 [1, 1, 51865] and the updated kv_cache_*

Integration notes

• Compute the log-mel input with the base model's WhisperFeatureExtractor (16 kHz mono, padded to 30 s). The tokenizer and generation config also come from the base model repository.
• Run encode once per window and bind its cross_kv_* outputs (492 MB) directly as decode inputs.
• Run decode one token at a time, including the prompt <|startoftranscript|><|zh|><|transcribe|><|notimestamps|> (50258, 50260, 50359, 50363) at positions 0–3; there is no prefill signature. EOS is 50257. Decode greedily with the base model's suppress_tokens and begin_suppress_tokens.
• Bind kv_cache_* (84 MB) as both input and output of decode so it is updated in place, and zero it once before the first call: a new buffer can hold NaN, which the attention mask does not hide.
• Set an XNNPACK weight cache file (xnnpack_weight_cache_path in Python, xnnPackWeightCachePath in Kotlin). It cut load time from 0.7–0.9 s to 0.05 s and peak RSS from 4.1 GB to 2.6 GB; the cache file is 1.55 GB.

How to use

pip install ai-edge-litert transformers librosa huggingface_hub
hf download litert-community/Breeze-ASR-26-LiteRT transcribe.py --local-dir .
python transcribe.py sample.wav --threads 8 --weight_cache breeze.xnnpack_cache

The script downloads the model on its first run and does not need PyTorch. On an 11.1 s TaigiSpeech test clip it prints:

沒事 沒事 取消了 不是我不小心按到的 不好意思 簡報 響了
[load 0.05 s | 1 window(s) | encode 4.00 s | decode 2.04 s, 26 steps, 79 ms/step | audio 11.1 s]

On Android, the Kotlin CompiledModel API (com.google.ai.edge.litert:litert:2.3.2, Kotlin 2.3 or newer) supports the same buffer binding; the A71 figures below come from such an app. No Kotlin log-mel frontend is included.

Performance

CPU (XNNPACK), weight cache on, greedy decoding:

Device	Threads	Encode (30 s window)	Decode step	Peak RSS
Intel Core i7-12700KF, Ubuntu on WSL2, ai-edge-litert 2.3.0	8	4.0 s	79–81 ms	2.6 GB
Samsung Galaxy A71 5G (Exynos 980, Android 13), LiteRT 2.3.2	4	21.4–23.6 s	612–648 ms	2.6 GB

Encode always processes a full 30 s window; decode time grows with the number of output tokens. The 11.1 s clip takes about 6 s end to end on the desktop. On the A71, 5.0 s and 6.1 s clips took 37.5 s and 33.7 s, so the phone suits offline transcription, not live captioning. On a Linux x86-64 server, one decode step took 47 ms, against 403 ms for the cache-free export_hf export with the same int8 recipe.

Accuracy

Agreement with the original model. On the full test split of TaigiSpeech (960 clips, 6 speakers, CC BY 4.0, no transcripts), compared with the base model in transformers (float32, greedy):

• 738 of 960 clips (76.9%, 95% CI 74.2–79.5%) give exactly the same tokens.
• 4.88% of characters differ (95% CI 4.10–5.68%), with no tendency toward longer or shorter output. Two clips on which the original model itself repeats a phrase until the limit are left out of this figure.

CER on government announcements. The paper's benchmark (30 Taigi announcements from the Executive Yuan, scored against Mandarin text) is not published, so the protocol was rebuilt from the same monthly packages: 37 announcements from February 2024 to April 2026, scored against the official Mandarin script.

System	Mean CER
Breeze-ASR-26 in transformers, float32	39.11%
This file	37.18%

The difference, −1.94 points (95% CI −5.90 to +0.41), is not significant; most of it comes from one clip on which the float32 model skipped most of the speech. Both are above the paper's 30.13%: the clips differ, the reference is the raw script rather than the paper's refined ground truth, and numerals are not normalized.

Output can differ slightly between CPU architectures: a clip that differs from the original on x86 matched it on the A71.

Limitations

• From the upstream card: the model writes Mandarin characters, not Taigi orthography, and was trained on synthetic speech, so accuracy may drop on strong accents, dialects and specialized terms.
• transcribe.py cuts long audio at fixed 30 s boundaries, so a word on a boundary can be lost or misheard. Split long recordings at pauses.
• Each window yields at most 252 output tokens. The language is fixed to <|zh|>, with no timestamps.
• Only the CPU path has been tested. On Android, plan for about 2.6 GB of memory and 3.1 GB of storage (model plus weight cache).

Training data

No training was done for this repository, and dynamic-range int8 needs no calibration data. According to its card, the base model was fine-tuned on about 10,000 hours of synthetic Taigi speech, starting from Whisper large-v2. The upstream card makes no separate statement about personally identifiable information; this conversion adds no data. The audio used for the checks above is not redistributed here.

Conversion

Converted with litert-torch 0.10.0 and ai-edge-quantizer 0.10.0 (dynamic_wi8_afp32) from the KV-cached Whisper modules proposed in google-ai-edge/litert-torch#1424 (branch). The proposal adds --whisper_kv_cache_max_len to export_hf; this file has the interface it produces with a value of 256. Follow the issue for a supported export command.

License and citation

Apache 2.0, the license of the base model. Whisper is released by OpenAI under the MIT license. The base model was built by the Speech AI Research Center at NYCU and MediaTek Research; if you use it, please cite:

@misc{lan2026breezetaigibenchmarksmodels,
      title={Breeze Taigi: Benchmarks and Models for Taiwanese Hokkien Speech Recognition and Synthesis},
      author={Yu-Siang Lan and Chia-Sheng Liu and Yi-Chang Chen and Po-Chun Hsu and Allyson Chiu and Shun-Wen Lin and Da-shan Shiu and Yuan-Fu Liao},
      year={2026},
      eprint={2603.19259},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2603.19259},
}