Automatic Speech Recognition
LiteRT-LM
LiteRT
VibeVoice
litertlm
on-device
edge
asr
speech-recognition
audio
bitnet
Instructions to use litert-community/VibeVoice-ASR-BitNet with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT-LM
How to use litert-community/VibeVoice-ASR-BitNet with LiteRT-LM:
# LiteRT-LM runs on various platforms (Android, iOS, Windows, Linux, macOS, IoT, Web/WASM) # and supports many APIs (C++, Python, Kotlin, Swift, JavaScript, Flutter). # For platform-specific integration guides, please refer to the official developer website: # https://ai.google.dev/edge/litert-lm # To try LiteRT-LM, the easiest way is to use our CLI tool. # 1. Install the LiteRT-LM CLI tool: pip install -U litert-lm # 2. Download and run this model locally: # See: https://ai.google.dev/edge/litert-lm/cli # A single .litertlm file in the repo is picked automatically; otherwise the CLI asks which one to run # (or pass its name right after the repo id). litert-lm run \ --from-huggingface-repo=litert-community/VibeVoice-ASR-BitNet \ --prompt="Write me a poem"
- LiteRT
How to use litert-community/VibeVoice-ASR-BitNet with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- VibeVoice
How to use litert-community/VibeVoice-ASR-BitNet with VibeVoice:
import torch, soundfile as sf, librosa, numpy as np from vibevoice.processor.vibevoice_processor import VibeVoiceProcessor from vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInference # Load voice sample (should be 24kHz mono) voice, sr = sf.read("path/to/voice_sample.wav") if voice.ndim > 1: voice = voice.mean(axis=1) if sr != 24000: voice = librosa.resample(voice, sr, 24000) processor = VibeVoiceProcessor.from_pretrained("litert-community/VibeVoice-ASR-BitNet") model = VibeVoiceForConditionalGenerationInference.from_pretrained( "litert-community/VibeVoice-ASR-BitNet", torch_dtype=torch.bfloat16 ).to("cuda").eval() model.set_ddpm_inference_steps(5) inputs = processor(text=["Speaker 0: Hello!\nSpeaker 1: Hi there!"], voice_samples=[[voice]], return_tensors="pt") audio = model.generate(**inputs, cfg_scale=1.3, tokenizer=processor.tokenizer).speech_outputs[0] sf.write("output.wav", audio.cpu().numpy().squeeze(), 24000) - Notebooks
- Google Colab
- Kaggle
README and manifest notes: plainer wording ("chunked by the runtime"; the 20 LibriSpeech clips named)
Browse files- README.md +2 -2
- litertlm_manifest.json +1 -1
README.md
CHANGED
|
@@ -104,7 +104,7 @@ If you downloaded this file with the litert-lm CLI's `--from-huggingface-repo` b
|
|
| 104 |
Notes:
|
| 105 |
|
| 106 |
- Greedy decoding (top_k 1) is what the vendor runtime does; the model emits plain text and stops on `<|im_end|>`.
|
| 107 |
-
- The encoder window is **30 s**. Longer clips are chunked by the
|
| 108 |
- KV budget 2048 tokens: 30 s of audio is 225 tokens plus the ~40-token prompt, leaving room for the transcript.
|
| 109 |
|
| 110 |
## Performance
|
|
@@ -120,7 +120,7 @@ Everything in this section above its last part was measured on earlier files (th
|
|
| 120 |
|
| 121 |
Init is the `Init time` the CLI prints; that value includes both `Init Total` and the phases inside it, so it counts engine creation twice ([LiteRT-LM #4057](https://github.com/google-ai-edge/LiteRT-LM/issues/4057)).
|
| 122 |
|
| 123 |
-
End-to-end on the
|
| 124 |
|
| 125 |
Galaxy S26 (SM-S942Q, Snapdragon SM8850, Adreno), `litert_lm_advanced_main` built from the litert-lm v0.16.1 tag, `--benchmark` with a 264-token text prompt, one reading per configuration (LM only):
|
| 126 |
|
|
|
|
| 104 |
Notes:
|
| 105 |
|
| 106 |
- Greedy decoding (top_k 1) is what the vendor runtime does; the model emits plain text and stops on `<|im_end|>`.
|
| 107 |
+
- The encoder window is **30 s**. Longer clips are chunked by the runtime (30 s windows, no overlap), which resets the encoder's context at each boundary; a 10 s window measured +1.6 pp WER on this set from those resets, which is why the shipped window is 30 s.
|
| 108 |
- KV budget 2048 tokens: 30 s of audio is 225 tokens plus the ~40-token prompt, leaving room for the transcript.
|
| 109 |
|
| 110 |
## Performance
|
|
|
|
| 120 |
|
| 121 |
Init is the `Init time` the CLI prints; that value includes both `Init Total` and the phases inside it, so it counts engine creation twice ([LiteRT-LM #4057](https://github.com/google-ai-edge/LiteRT-LM/issues/4057)).
|
| 122 |
|
| 123 |
+
End-to-end on the 20 LibriSpeech clips above (engine already loaded, one conversation per clip): real-time factor **0.24 on CPU** and **0.19 with the LM on Metal**, i.e. a 10 s clip transcribes in about 2 s.
|
| 124 |
|
| 125 |
Galaxy S26 (SM-S942Q, Snapdragon SM8850, Adreno), `litert_lm_advanced_main` built from the litert-lm v0.16.1 tag, `--benchmark` with a 264-token text prompt, one reading per configuration (LM only):
|
| 126 |
|
litertlm_manifest.json
CHANGED
|
@@ -83,7 +83,7 @@
|
|
| 83 |
"platform_notes": [
|
| 84 |
"Audio only through the runtime's generic audio path: send the clip as an audio content item (wav/mp3/flac; any sample rate — the runtime resamples to 24 kHz) plus the text 'This is a X.XX seconds audio, please transcribe it.' (X.XX = clip length); with no text the file's template sends 'Please transcribe it.' (+0.9 pp WER on the 20 LibriSpeech clips)",
|
| 85 |
"audio_backend must be cpu (--audio_backend=cpu / audio_backend=CPU()); the LM may run on gpu",
|
| 86 |
-
"The encoder window is 30 s; longer clips are chunked by the
|
| 87 |
"Accuracy: 20 LibriSpeech dev-clean clips (448 words): WER 2.68 % on macOS CPU and 2.46 % LM-on-GPU (fp32 PyTorch reference 2.68 %), 3.12 % Galaxy S26 CPU, 2.46 % Galaxy S26 LM-on-GPU; vendor card Libri-clean 2.41 % (full test set, VibeASR.cpp)",
|
| 88 |
"Peak private footprint on Galaxy S26: 3227 MB CPU/CPU, 2268 MB LM-on-GPU (--report_peak_memory_footprint, 5.9 s clip)",
|
| 89 |
"2026-11-17 GPU graph (weights unchanged): the prefill/decode graph has the GPU composite form (28 odml.cache_update composites for the KV cache and 56 attention composites; the previous file had none); every weight and every section outside that graph has the same size and sha256 as in the previous file (Hub revision 0d3aca6f). This file was tested on litert-lm 0.19.0-0.19.2 (macOS) and litert_lm_advanced_main v0.19.0 (Galaxy S26, Pixel 11 Pro). On macOS with litert-lm 0.19.2 this file and the previous file gave the same transcripts of the 20 clips: 18/448 word errors (4.02 %) with both files on the CPU with 1 thread, 11/448 (2.46 %) with both files with the LM on the GPU. The figures in the notes above, peak_ram_mb, the macOS recommendation and the measured rows dated 2026-09-04 were measured on earlier files."
|
|
|
|
| 83 |
"platform_notes": [
|
| 84 |
"Audio only through the runtime's generic audio path: send the clip as an audio content item (wav/mp3/flac; any sample rate — the runtime resamples to 24 kHz) plus the text 'This is a X.XX seconds audio, please transcribe it.' (X.XX = clip length); with no text the file's template sends 'Please transcribe it.' (+0.9 pp WER on the 20 LibriSpeech clips)",
|
| 85 |
"audio_backend must be cpu (--audio_backend=cpu / audio_backend=CPU()); the LM may run on gpu",
|
| 86 |
+
"The encoder window is 30 s; longer clips are chunked by the runtime with a context reset at each 30 s boundary",
|
| 87 |
"Accuracy: 20 LibriSpeech dev-clean clips (448 words): WER 2.68 % on macOS CPU and 2.46 % LM-on-GPU (fp32 PyTorch reference 2.68 %), 3.12 % Galaxy S26 CPU, 2.46 % Galaxy S26 LM-on-GPU; vendor card Libri-clean 2.41 % (full test set, VibeASR.cpp)",
|
| 88 |
"Peak private footprint on Galaxy S26: 3227 MB CPU/CPU, 2268 MB LM-on-GPU (--report_peak_memory_footprint, 5.9 s clip)",
|
| 89 |
"2026-11-17 GPU graph (weights unchanged): the prefill/decode graph has the GPU composite form (28 odml.cache_update composites for the KV cache and 56 attention composites; the previous file had none); every weight and every section outside that graph has the same size and sha256 as in the previous file (Hub revision 0d3aca6f). This file was tested on litert-lm 0.19.0-0.19.2 (macOS) and litert_lm_advanced_main v0.19.0 (Galaxy S26, Pixel 11 Pro). On macOS with litert-lm 0.19.2 this file and the previous file gave the same transcripts of the 20 clips: 18/448 word errors (4.02 %) with both files on the CPU with 1 thread, 11/448 (2.46 %) with both files with the LM on the GPU. The figures in the notes above, peak_ram_mb, the macOS recommendation and the measured rows dated 2026-09-04 were measured on earlier files."
|