luweigen/vibevoice-asr-streaming-android

VibeVoice-ASR-Streaming-1.5B — models prepared for Android

Derived from microsoft/VibeVoice-ASR-Streaming-1.5B (MIT) for the native Android app in the vibevoice-android project. Files are downloaded by the app and checked against sha256 values compiled into the APK. See the original model card for intended use, limitations and the license.

File	What	Runs on
encoder.ns_ctx.onnx + encoder.ns_ctx_qnn.bin	Speech encoder (acoustic + semantic tokenizer encoders and connectors), audio[1,83200] → feats[1,26,1536], fp16, QNN HTP context binary compiled offline with onnxruntime-qnn 2.7.0 (QNN 2.51). RMSNorm inputs are pre-scaled by powers of two (eps scaled to match) so that fp16 on the HTP does not flush eps to zero	Snapdragon 8 Elite (SM8750, HTP v79) only; device needs qnn-runtime ≥ 2.51
encoder.static.onnx + encoder.static.onnx.data	The same encoder, fp32, static shapes, FFN as 1×1 convolutions	Any device, ONNX Runtime CPU EP
vv-enc.c0of6.fp16.tflite … vv-enc.c5of6.fp16.tflite	The same speech encoder as six LiteRT (TFLite) segments with fp16 weights, run one after another on the GPU (LiteRT 2.3.2 GPU accelerator, OpenCL). Segment 0 takes audio[1,83200]; every segment advances both tokenizer branches (two tensors in, two out, output i → input i of the next segment); segment 5 adds the two connectors and outputs feats[1,26,1536]. Mathematically equivalent rewrites for the mobile GPU: layer scales folded into the preceding weights, RMSNorm computed per frame after dividing by the frame's max |x| (with a small fp16-normal offset), and the residual stream of stages 0–5 multiplied by 1024 (undone before stage 6) so that small activations are not flushed as fp16 subnormals. Segment 0 (stages 0–2) must run with fp32 GPU precision; the rest in fp16	Tested on Adreno 630 (OnePlus 6, Snapdragon 845): 1.9 s per window, cosine 0.9995–0.9997 to fp32; same CER as the CPU encoder
lm-q4_0.gguf	The Qwen2-1.5B language model as GGUF (Q4_0), Qwen2 BPE tokenizer included. The token embedding table is the model's own; no other changes	llama.cpp (CPU, OpenCL, Hexagon)
special_embeds.bin	float32 [2, 1536]: embeddings of the speech start / end tokens that wrap each chunk's audio features	—

The acoustic latent uses the mean of the VAE posterior (no sampling), which matches the official sampling mode in WER (docs/models.md of the project). Streaming protocol: every 2.93 s of audio (window 3.47 s incl. lookahead), prefill [speech_start, 26 features, speech_end], decode greedily to <|text_chunk_end|>, feed it back; the KV cache is kept across chunks.