How to use from the
Use from the
VibeVoice library
import torch, soundfile as sf, librosa, numpy as np
from vibevoice.processor.vibevoice_processor import VibeVoiceProcessor
from vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInference

# Load voice sample (should be 24kHz mono)
voice, sr = sf.read("path/to/voice_sample.wav")
if voice.ndim > 1: voice = voice.mean(axis=1)
if sr != 24000: voice = librosa.resample(voice, sr, 24000)

processor = VibeVoiceProcessor.from_pretrained("litert-community/VibeVoice-ASR-BitNet")
model = VibeVoiceForConditionalGenerationInference.from_pretrained(
    "litert-community/VibeVoice-ASR-BitNet", torch_dtype=torch.bfloat16
).to("cuda").eval()
model.set_ddpm_inference_steps(5)

inputs = processor(text=["Speaker 0: Hello!\nSpeaker 1: Hi there!"],
                   voice_samples=[[voice]], return_tensors="pt")
audio = model.generate(**inputs, cfg_scale=1.3,
                       tokenizer=processor.tokenizer).speech_outputs[0]
sf.write("output.wav", audio.cpu().numpy().squeeze(), 24000)

Measured on device with an earlier file (edge-compat): Galaxy S26 Β· LiteRT-LM 0.16.1 Β· GPU Β· decode 36.1 tok/s Β· prefill 653 tok/s Β· TTFT 430 ms Β· all 1243 ops delegated (2026-09-04); Mac Studio M4 Max Β· LiteRT-LM 0.16.0 Β· GPU Β· decode 138.6 tok/s Β· prefill 2045 tok/s Β· TTFT 134 ms (2026-09-04); Galaxy S26 Β· LiteRT-LM 0.16.1 Β· CPU Β· decode 35.4 tok/s Β· prefill 163 tok/s Β· TTFT 1.65 s (2026-09-04). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/vibevoice-asr-bitnet/CARD.md

VibeVoice-ASR-BitNet β€” LiteRT-LM

microsoft/VibeVoice-ASR-BitNet converted to the LiteRT-LM (.litertlm) format for on-device speech recognition with Google's LiteRT-LM runtime. Audio in, text out, in one file: the runtime decodes the clip, runs the file's audio encoder, and the LLM transcribes β€” no host-side feature extraction.

On 2026-11-17 the file in this repo was replaced with a file whose prefill/decode graph is rewritten for the GPU, with every weight unchanged. On a Galaxy S26 GPU the new file decodes 51.2 tokens/s after a 1,024-token prefill, against 46.7 tokens/s for the file it replaced (LiteRT-LM v0.19.0; see the end of Performance). The new file was run only with LiteRT-LM 0.19.0 to 0.19.2; earlier runtimes were not tried with it.

VibeVoice-ASR-BitNet is Microsoft's edge variant of VibeVoice-ASR: the same Οƒ-VAE acoustic + semantic tokenizers (24 kHz β†’ 7.5 latent frames/s) feeding a Qwen2.5-1.5B-shaped language model trained with BitNet b1.58 ternary weights, multilingual (en, zh, fr, it, ko, pt, vi), MIT-licensed. This is the first audio-in .litertlm in this collection.

File Recipe Size
VibeVoice-ASR-BitNet.litertlm LM: per-tensor ternary weights stored as int4 blockwise-128 (exact), int8 embedding Β· audio encoder: int8 dynamic linears, fp32 convs, 30 s window 1.98 GB

Sections: prefill/decode 793 MB, embedder 237 MB, audio encoder 951 MB, tokenizer.

Correctness

20 LibriSpeech dev-clean clips (448 words, 2–29 s), greedy decoding, the vendor's prompt (This is a X.XX seconds audio, please transcribe it.), WER after uppercasing and stripping punctuation:

Configuration WER
PyTorch fp32 reference (ternarized LM, deterministic latents) 2.68 % (12/448)
LiteRT-LM 0.16.1, Apple M4 Max, CPU 2.68 % (12/448) β€” transcripts identical to the reference
LiteRT-LM 0.16.1, Apple M4 Max, LM on Metal GPU, audio on CPU 2.46 % (11/448)
LiteRT-LM v0.16.1 CLI, Galaxy S26 (SM-S942Q, Snapdragon SM8850), CPU 3.12 % (14/448)
LiteRT-LM v0.16.1 CLI, Galaxy S26, LM on OpenCL GPU, audio on CPU 2.46 % (11/448)*

* measured on the preceding build of this file: identical LM sections, the audio encoder differing only in its normalisation arithmetic, whose CPU transcripts are the same on every platform (the re-run on the shipped build agreed on its first two clips before the phone dropped off USB).

The Android CPU row differs from the Mac by one clip ("On the general principles…" β†’ "Under general principles…"), an int8 dynamic-range numerics flip on Arm (a Pixel 8a produced the identical transcript); the GPU LM row matches the Mac GPU row word for word. For reference, Microsoft reports 2.41 % on the full LibriSpeech test-clean set with their VibeASR.cpp runtime.

Sending the clip without the duration sentence works (the file's template then sends "Please transcribe it.") at 3.57 % on the same set β€” include the duration for best results.

The table and the paragraphs above were measured on earlier files. The new file was compared with the file it replaced on the same 20 clips on a Mac with litert-lm 0.19.2 (2026-11-16, greedy decoding, the duration sentence in the prompt, audio encoder on the CPU). On the CPU with 1 thread, the new file gave the same transcript as the previous file for all 20 clips, byte for byte (18/448 word errors, 4.02 %, with both files). With the LM on the Mac GPU, the two files also gave the same transcripts for all 20 clips (11/448, 2.46 %, with both files). The CPU count of 18/448 is higher than the 12/448 of the table's Mac CPU row (LiteRT-LM 0.16.1). Both files give it, and no run on this card shows which difference between the two measurements accounts for it.

Usage

The audio goes in as an audio content item; any container/sample rate the runtime's decoder reads (wav, mp3, flac β€” it resamples to 24 kHz). The audio encoder must run on the CPU (audio_backend), the LM may run on CPU or GPU. On Android the GPU was the faster of the two with the new file on a Galaxy S26 (numbers at the end of Performance); the two blocks below run the LM on the CPU.

Python (pip install litert-lm-api; this block was run as written on 0.16.1 and 0.17.1 with earlier files). On 2026-11-17 it was run as written on 0.19.2 on a Mac, with a clip's path in place of /abs/path/clip.wav: once with the new file and once with the file it replaced. Both runs printed the same text.

import litert_lm
from litert_lm import Message, Contents, Content
from litert_lm.interfaces import CPU, GPU

engine = litert_lm.Engine("VibeVoice-ASR-BitNet.litertlm", backend=CPU(), audio_backend=CPU())
dur = 5.86  # clip length in seconds
conv = engine.create_conversation(sampler_config=litert_lm.SamplerConfig(top_k=1, top_p=1.0, temperature=0.0),
                                  max_output_tokens=256)

def response_text(resp):
    # litert-lm-api 0.15-0.17.1 return a dict of content parts; later builds return a Message whose str() is its text
    if type(resp) is dict:
        return "".join(p.get("text", "") for p in resp.get("content", []) if isinstance(p, dict))
    return str(resp)

resp = conv.send_message(Message.user(Contents.of([
    Content.AudioFile("/abs/path/clip.wav"),
    Content.Text(f"This is a {dur:.2f} seconds audio, please transcribe it."),
])))
print(response_text(resp))
conv.close()
# CLI (litert_lm_advanced_main / litert-lm run); the [audio:…] tag goes first
./litert_lm_advanced_main --backend=cpu --audio_backend=cpu --model_path=VibeVoice-ASR-BitNet.litertlm \
  --input_prompt='[audio:/path/clip.wav] This is a 5.86 seconds audio, please transcribe it.'

If you download this file with the litert-lm command-line tool (PyPI litert-lm, not the litert_lm_advanced_main program) and it stops right after the download with no error message of its own (a bus error; exit status 138 on a Mac) or with an error log that contains too large for file size, the downloaded file may be incomplete. The litert-lm CLI (0.14.0 to 0.19.2, and litert-lm-nightly 0.20.0.dev20261112) can keep an incomplete download in ~/.litert-lm/cache/huggingface/litert-community/VibeVoice-ASR-BitNet/ and uses it again on later runs without checking its size. Delete that file and run the command again (after an import, run the import again). This was measured with other models' files, not with this one.

If you downloaded this file with the litert-lm CLI's --from-huggingface-repo before it was replaced on 2026-11-17 (11:58 UTC), delete it from ~/.litert-lm/cache/huggingface/litert-community/VibeVoice-ASR-BitNet/, then run your command again (after an import, run the import again). Without deleting it, the litert-lm CLI (0.14.0 to 0.19.2) keeps running the previous file: it uses a file it finds there without checking it against this repo. The previous file stays at revision 0d3aca6.

Notes:

  • Greedy decoding (top_k 1) is what the vendor runtime does; the model emits plain text and stops on <|im_end|>.
  • The encoder window is 30 s. Longer clips are chunked by the runtime (30 s windows, no overlap), which resets the encoder's context at each boundary; a 10 s window measured +1.6 pp WER on this set from those resets, which is why the shipped window is 30 s.
  • KV budget 2048 tokens: 30 s of audio is 225 tokens plus the ~40-token prompt, leaving room for the transcript.

Performance

Everything in this section above its last part was measured on earlier files (the tables on 2026-09-04). The new file's speeds were measured on a Galaxy S26 (GPU and CPU) and on a Pixel 11 Pro (GPU); they are in the last part of this section. This card has no Mac speed for the new file.

litert-lm benchmark (litert-lm 0.16.0), Apple M4 Max, -p 256 -d 256 --runs 3 --cache no, text prompt (LM only β€” the audio encoder is a fixed cost per 30 s window: 1.3–1.7 s on the M4 Max CPU at 8 threads), quiet machine, β‰₯ 300 s rest before the GPU reading:

Backend Prefill (256) Decode TTFT Init
GPU (Metal) 2045 tok/s 138.6 tok/s 0.13 s 1.4 s
CPU 357 tok/s 59.3 tok/s 0.81 s β€”

Init is the Init time the CLI prints; that value includes both Init Total and the phases inside it, so it counts engine creation twice (LiteRT-LM #4057).

End-to-end on the 20 LibriSpeech clips above (engine already loaded, one conversation per clip): real-time factor 0.24 on CPU and 0.19 with the LM on Metal, i.e. a 10 s clip transcribes in about 2 s.

Galaxy S26 (SM-S942Q, Snapdragon SM8850, Adreno), litert_lm_advanced_main built from the litert-lm v0.16.1 tag, --benchmark with a 264-token text prompt, one reading per configuration (LM only):

Backend Prefill (264) Decode TTFT
GPU (OpenCL) 652 tok/s 36.1 tok/s 0.43 s
CPU 163 tok/s 35.4 tok/s 1.65 s

On the phone a 5–20 s clip transcribes in roughly 7–10 s of wall-clock from a cold process (CPU), including engine load; peak private footprint 3.2 GB (CPU) / 2.3 GB (LM on GPU).

Galaxy S26 and Pixel 11 Pro: the file of 2026-11-17

Each run prefills 1,024 tokens and generates 256 tokens, three times in one process (three iterations), with a text prompt (LM only) and a context of 2,048 tokens. The program is litert_lm_advanced_main built from the LiteRT-LM v0.19.0 source (commit 771b06e1; the tag's own commit after it changes only Package.swift and README.md), run with --backend=gpu or --backend=cpu and --async=false --benchmark --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=3 --max_num_tokens=2048 on 2026-11-17. The CPU runs used the program's default of 4 threads. Each set of four runs (one set per phone on the GPU, two sets on the Galaxy S26 CPU) went in the order previous, new, new, previous. Each run is a new process. Each value is one run's mean of iterations 2–3, so it leaves out iteration 1, the first pass after the engine starts.

GPU:

Phone Prefill, tok/s (previous, new, new, previous) Decode, tok/s (previous, new, new, previous) New / previous, runs 2 and 1 New / previous, runs 3 and 4
Galaxy S26 689.25, 710.56, 712.12, 686.85 46.85, 51.08, 51.34, 46.59 prefill Γ—1.031, decode Γ—1.090 prefill Γ—1.037, decode Γ—1.102
Pixel 11 Pro 587.37, 610.56, 605.76, 590.42 13.87, 15.50, 16.38, 14.42 prefill Γ—1.039, decode Γ—1.118 prefill Γ—1.026, decode Γ—1.136

Galaxy S26 CPU (XNNPACK), two sets of four runs:

Set Prefill, tok/s (previous, new, new, previous) Decode, tok/s (previous, new, new, previous) New / previous, runs 2 and 1 New / previous, runs 3 and 4
first 186.24, 184.84, 189.38, 195.21 35.80, 35.55, 36.05, 36.92 prefill Γ—0.992, decode Γ—0.993 prefill Γ—0.970, decode Γ—0.977
second 183.99, 185.72, 196.04, 200.15 35.99, 36.46, 37.03, 36.87 prefill Γ—1.009, decode Γ—1.013 prefill Γ—0.979, decode Γ—1.004

The mean of each file's two runs:

Phone and backend New file: prefill / decode / time to first token Previous file: prefill / decode / time to first token
Galaxy S26, GPU 711.34 tok/s / 51.21 tok/s / 1.46 s 688.05 tok/s / 46.72 tok/s / 1.51 s
Pixel 11 Pro, GPU 608.16 tok/s / 15.94 tok/s / 1.75 s 588.90 tok/s / 14.14 tok/s / 1.81 s
Galaxy S26, CPU, first set 187.11 tok/s / 35.80 tok/s / 5.54 s 190.73 tok/s / 36.36 tok/s / 5.42 s
Galaxy S26, CPU, second set 190.88 tok/s / 36.75 tok/s / 5.41 s 192.07 tok/s / 36.43 tok/s / 5.38 s

On the GPU the new file was faster in both pairs of runs on both phones. On the CPU the two sets do not agree. In the first set the new file was 0.7 % to 3.0 % slower in all four ratios. Two of them, runs 3 and 4 (3.0 % in prefill, 2.3 % in decode), are larger than the difference between the new file's own two runs of that set (2.4 % in prefill: 184.84 and 189.38 tok/s; 1.4 % in decode: 35.55 and 36.05 tok/s). In the second set it was 0.4 % to 1.3 % faster in three ratios and 2.1 % slower in one (prefill, runs 3 and 4), which is smaller than the 5.3 % by which the new file's own two runs of that set differ in prefill (185.72 and 196.04 tok/s). So this card claims no CPU speed change in either direction.

On the Galaxy S26, run the LM on the GPU. There the new file ran at 711.34 tok/s prefill and 51.21 tok/s decode on the GPU, against 190.88 and 36.75 tok/s on the CPU in the second set (187.11 and 35.80 in the first), with the same program and workload on the same day. The GPU was also the faster one in iteration 1: 714.37 and 716.12 tok/s prefill and 49.57 and 49.59 tok/s decode in the new file's two GPU runs, against 338.06 to 344.08 and 44.19 to 44.72 tok/s in its four CPU runs in the tables. The Pixel 11 Pro CPU was not run with the new file. The audio encoder stays on the CPU in either case.

Every GPU speed run started at Android's thermal status 0. On the Galaxy S26 (SM-S942Q) the GPU driver's thermal level (0 = no limit) reached 1 at most during the iterations of the GPU runs. On the Pixel 11 Pro, Android's thermal status stayed at 0 in every speed run. The CPU runs in the tables started at thermal status 0, except the first run of the first set, which started at 1. Before each CPU set, one more run of the new file was made and is not counted. A system app (com.google.android.gms) was in the foreground when each Galaxy S26 run started.

On both phones, the new file and the file it replaced gave the same transcripts of two LibriSpeech clips (1272-128104-0000 and 1272-128104-0001), byte for byte and with no word error against the reference texts (17 and 10 words): Mister quilter is the apostle of the middle classes, and we are glad to welcome his gospel. and Nor is mister quilter's manner less interesting than his matter. Each clip went in with its duration sentence, with the LM on the GPU, the audio encoder on the CPU and greedy decoding (--sampler_top_k=1).

Conversion notes

Converted with litert-torch 0.9.3/0.9.4, ai-edge-quantizer 0.9.0, litert-lm-builder 0.16.1, transformers 5.14.1 (native vibevoice_asr). Scripts and the full recipe: hf-to-litertlm vibevoice_asr_work/.

  • Ternarization is applied at conversion, per tensor, exactly as the vendor runtime does. The Hub checkpoint stores fp32 latent weights; VibeASR.cpp quantizes the seven projections per LM layer with a per-tensor absmean scale at GGUF-conversion time. The same values are stored here as int4 blockwise-128 min-max, which represents a per-tensor ternary exactly (every block is {βˆ’Ξ±, 0, +Ξ±}); the embedding/head stays int8.
  • Audio front-end folded into the encoder graph. Input is raw 24 kHz PCM framed by the runtime (3200 samples = one latent frame); the vendor's βˆ’25 dBFS RMS normalisation and peak clip run in-graph. Acoustic latents are the mean (no sampling noise), as in the vendor runtime.
  • Tokenizer: the upstream tokenizer.json (Qwen2) has no string for the speech-marker ids the model was trained with; <|object_ref_start|> / <|object_ref_end|> / <|box_start|> are added as special tokens on ids 151646/151647/151648 so the runtime can emit them around the audio embeddings. Ordinary text tokenizes identically.
  • Prompt: the vendor's system prompt is baked into the file's template; the user turn renders as <|object_ref_start|> + audio embeddings + <|object_ref_end|> + newline + your text.
  • GPU: the LM runs on Metal (macOS) and on the Android OpenCL delegate; the audio encoder is CPU-only β€” on Metal/WebGPU the 30 s window exceeds the delegate's 65 535-workgroup dispatch limit, and on Adreno the fp16 delegate path returns an empty transcript. Keep audio_backend on CPU.

2026-11-17 β€” GPU graph (weights unchanged)

The prefill/decode graph now has the GPU form: 28 odml.cache_update composites for the KV cache and 56 attention composites. The previous file had neither; a check of its graph counted 56 DYNAMIC_UPDATE_SLICE and 56 BATCH_MATMUL ops instead. The prefill lengths (32, 128 and 512 tokens), the KV-cache length (2,048) and the weight quantization read the same before and after.

Only the prefill/decode graph section differs from the previous file: it went from 20,937,360 B to 21,289,304 B. The six other sections (metadata, tokenizer, embedder graph, audio encoder and the two weight sections) have the same size and sha256 as in the previous file. The file went from sha256 498ad85c… (1,983,053,824 B) to 7433f1a9… (1,983,414,272 B).

With litert-lm 0.19.0, 0.19.1 and 0.19.2 on a Mac, on the CPU and on the GPU, the new file and the file it replaced both transcribed two test clips, to the same text byte for byte. The Mac comparison on 20 clips is under Correctness and the phone runs are in the last part of Performance.

Apart from the parts that name the file of 2026-11-17, this card's measurements of the file were made with its earlier versions.

Known runtime issues

External weights (since 2026-11-09). In VibeVoice-ASR-BitNet.litertlm, the weights sit in TFLiteWeights sections at the end of the file (LiteRT's external weights) instead of inside the model graph. On a Pixel 8a, the GPU engine of the v0.18.0 and v0.18.1 Android builds crashes (SIGSEGV) on a Qwen3-0.6B file with external weights (LiteRT-LM#3986). The earlier file with the weights inside the graph stays at revision 193b2d5.

From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders this file's chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with Failed to apply template: undefined value. This happens in two places: when the preface is prefilled at creation (prefillPrefaceOnInit = true in Kotlin, prefillPrefaceOnInit: true on the web, SetPrefillPrefaceOnInit(true) in C++; the default is false), where creating the conversation fails with that error, and in the preface render (renderPrefaceIntoString() in Kotlin and Swift, RenderPrefaceIntoString() in C++, litert_lm_conversation_render_preface_to_string in the C API). The same happens with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on this file's template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.

From the second message of a conversation on, LiteRT-LM puts the previous answer's stop token into the KV cache with a zero embedding, because this file holds the embedding table in its own section and its prefill/decode graph takes embeddings as input (read in the source of LiteRT-LM 0.16.0, 0.18.1 and 0.19.1; measured with 0.18.1 on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on this file). The first message of a conversation is not affected, so send one clip per conversation, as the blocks under Usage do.

On LiteRT-LM 0.18.1 and 0.19.1, once a message fails because it does not fit in the context window, every later message in that conversation fails as well (the next one with Task failed with state: 7). Start a new conversation after such an error. This was measured with other models' files, not with this one.

License and changes

Distributed under the MIT License (inherited from the base model; see LICENSE). Changes from the original work: weights ternarized (LM) and converted from safetensors fp32 to LiteRT flatbuffers with the quantization described above; the VAE decoder (synthesis half) is not included; three special tokens added to the tokenizer; chat/prompt template and audio-preprocessing parameters embedded as LiteRT-LM metadata.

Downloads last month
354
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for litert-community/VibeVoice-ASR-BitNet

Quantized
(6)
this model

Collections including litert-community/VibeVoice-ASR-BitNet