luweigen/parakeet-tdt-0.6b-v3-android

Parakeet-TDT-0.6B-v3 — ONNX files, a Qualcomm NPU encoder and a mobile-GPU encoder for Android

Source

• Original model: nvidia/parakeet-tdt-0.6b-v3 (CC-BY-4.0), NVIDIA's multilingual Parakeet TDT 0.6B v3 (25 European languages).
• ONNX export: istupakov/parakeet-tdt-0.6b-v3-onnx (CC-BY-4.0), revision 8f23f0c03c8761650bdb5b40aaf3e40d2c15f1ce, made for onnx-asr.
• This repository is used by the native Android app of the vibevoice-android project. The app downloads the files and checks each one against a sha256 value compiled into the APK. See the original model card for intended use, limitations and the license terms.

Files

File	What	Origin
nemo128.onnx	Audio → 128-bin log-mel features (with per-feature normalization)	Unchanged mirror of the istupakov export
encoder-model.onnx + encoder-model.onnx.data	FastConformer encoder, fp32, dynamic shapes: audio_signal[B,128,T], length[B] → outputs[B,1024,T/8]	Unchanged mirror
decoder_joint-model.onnx	TDT prediction network + joint network, one step per call	Unchanged mirror
vocab.txt	SentencePiece vocabulary (token id, <blk> last)	Unchanged mirror
parakeet-enc-609.c0of3.fp16.tflite, …c1of3…, …c2of3…	The same encoder as three LiteRT (TFLite) segments with fp16 weights (8 layers each), fixed input [1,128,609] (a 6.08 s window: 4 s left context + 1 s chunk + 1 s right context) → [1,1024,77], run one after another on the GPU (LiteRT 2.3.2 GPU accelerator, OpenCL). Built by loading the istupakov encoder weights into transformers' ParakeetEncoder (with scale_input=False, matching the original) and converting with litert-torch; no attention mask (the window is always full); LayerNorm computed per frame after dividing by the frame's max |x| (mathematically equivalent, avoids fp16 overflow)	Derived from encoder-model.onnx
parakeet-enc-601_ctx.onnx + parakeet-enc-601_ctx_qnn.bin	The same encoder with a fixed input [1,128,601] (a 6.0 s window: 4 s left context + 0.5 s chunk + 1.5 s right context) → [1,1024,76]. length was replaced by a constant and the masks folded with ONNX Runtime's basic graph optimizations, so the whole graph runs on the HTP; then compiled offline to a QNN HTP context (fp16) with onnxruntime-qnn 2.7.0 / QNN 2.51	Derived from encoder-model.onnx

The NPU encoder matches the fp32 encoder to cosine 0.990–0.999 per window (fp16 error accumulates over the 24 layers); on a Finnish FLEURS test stream the word error rate is unchanged (14.6 % vs 14.6 % on the CPU).

Where the files apply

• parakeet-enc-601_ctx*: Snapdragon 8 Elite (SM8750, Hexagon HTP v79) only, with a QNN runtime ≥ 2.51 and ONNX Runtime with the QNN execution provider. The context binary is specific to this SoC and QNN version. The input length is fixed: the caller must feed exactly 601 log-mel frames (96 000 samples at 16 kHz) per call.
• parakeet-enc-609.c*of3.fp16.tflite: phones with an Adreno GPU and OpenCL, with LiteRT 2.3.2 (libLiteRt.so + libLiteRtClGlAccelerator.so). Tested on Adreno 630 (OnePlus 6, Snapdragon 845): 0.40 s per window, cosine 0.9999 to fp32, word error rate 14.5 % vs 14.2 % on the CPU (1 s chunks instead of 4 s), speech-to-text latency 2.5 s instead of 6.7 s. Feed exactly 609 log-mel frames (97 280 samples at 16 kHz) per call.
• nemo128.onnx, encoder-model.onnx(.data), decoder_joint-model.onnx, vocab.txt: any device with ONNX Runtime (CPU EP); the app uses them on phones without that NPU, and nemo128.onnx / the decoder / vocab.txt together with the NPU encoder.
• 16 kHz mono audio. The model is an offline model; the app streams it with fixed windows and LocalAgreement.

Usage and code

To be added.