Automatic Speech Recognition
LiteRT-LM
LiteRT
VibeVoice
litertlm
on-device
edge
asr
speech-recognition
audio
bitnet
Instructions to use litert-community/VibeVoice-ASR-BitNet with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT-LM
How to use litert-community/VibeVoice-ASR-BitNet with LiteRT-LM:
# LiteRT-LM runs on various platforms (Android, iOS, Windows, Linux, macOS, IoT, Web/WASM) # and supports many APIs (C++, Python, Kotlin, Swift, JavaScript, Flutter). # For platform-specific integration guides, please refer to the official developer website: # https://ai.google.dev/edge/litert-lm # To try LiteRT-LM, the easiest way is to use our CLI tool. # 1. Install the LiteRT-LM CLI tool: pip install -U litert-lm # 2. Download and run this model locally: # See: https://ai.google.dev/edge/litert-lm/cli # A single .litertlm file in the repo is picked automatically; otherwise the CLI asks which one to run # (or pass its name right after the repo id). litert-lm run \ --from-huggingface-repo=litert-community/VibeVoice-ASR-BitNet \ --prompt="Write me a poem"
- LiteRT
How to use litert-community/VibeVoice-ASR-BitNet with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- VibeVoice
How to use litert-community/VibeVoice-ASR-BitNet with VibeVoice:
import torch, soundfile as sf, librosa, numpy as np from vibevoice.processor.vibevoice_processor import VibeVoiceProcessor from vibevoice.modular.modeling_vibevoice_inference import VibeVoiceForConditionalGenerationInference # Load voice sample (should be 24kHz mono) voice, sr = sf.read("path/to/voice_sample.wav") if voice.ndim > 1: voice = voice.mean(axis=1) if sr != 24000: voice = librosa.resample(voice, sr, 24000) processor = VibeVoiceProcessor.from_pretrained("litert-community/VibeVoice-ASR-BitNet") model = VibeVoiceForConditionalGenerationInference.from_pretrained( "litert-community/VibeVoice-ASR-BitNet", torch_dtype=torch.bfloat16 ).to("cuda").eval() model.set_ddpm_inference_steps(5) inputs = processor(text=["Speaker 0: Hello!\nSpeaker 1: Hi there!"], voice_samples=[[voice]], return_tensors="pt") audio = model.generate(**inputs, cfg_scale=1.3, tokenizer=processor.tokenizer).speech_outputs[0] sf.write("output.wav", audio.cpu().numpy().squeeze(), 24000) - Notebooks
- Google Colab
- Kaggle
litertlm_manifest.json: VibeVoice-ASR-BitNet.litertlm's sha256, size and sections after the move to external weights; min runtime 0.19.0 and the Android GPU known issue (LiteRT-LM#3986)
Browse files- litertlm_manifest.json +19 -7
litertlm_manifest.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
{
|
| 2 |
"manifest_schema": "0.1.3",
|
| 3 |
"repo": "litert-community/VibeVoice-ASR-BitNet",
|
| 4 |
-
"generated": "2026-
|
| 5 |
"generator": "make_manifest.py",
|
| 6 |
"model": {
|
| 7 |
"display_name": "VibeVoice-ASR-BitNet (speech-to-text LLM, 24 kHz audio in)",
|
|
@@ -22,8 +22,8 @@
|
|
| 22 |
"variants": [
|
| 23 |
{
|
| 24 |
"file": "VibeVoice-ASR-BitNet.litertlm",
|
| 25 |
-
"sha256": "
|
| 26 |
-
"size_bytes":
|
| 27 |
"sections": [
|
| 28 |
{
|
| 29 |
"type": "LlmMetadataProto",
|
|
@@ -35,18 +35,28 @@
|
|
| 35 |
},
|
| 36 |
{
|
| 37 |
"type": "TFLiteModel",
|
| 38 |
-
"size_bytes":
|
| 39 |
"model_type": "tf_lite_embedder"
|
| 40 |
},
|
| 41 |
{
|
| 42 |
"type": "TFLiteModel",
|
| 43 |
-
"size_bytes":
|
| 44 |
"model_type": "tf_lite_prefill_decode"
|
| 45 |
},
|
| 46 |
{
|
| 47 |
"type": "TFLiteModel",
|
| 48 |
"size_bytes": 951122160,
|
| 49 |
"model_type": "tf_lite_audio_encoder_hw"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
}
|
| 51 |
],
|
| 52 |
"quantization": "LM: per-tensor ternary weights (VibeASR.cpp absmean, applied at conversion) stored as int4 blockwise-128 min-max (exact for a ternary tensor), int8 embedding; audio encoder: int8 dynamic on linears, fp32 convs, 30 s (225-frame) window",
|
|
@@ -55,7 +65,7 @@
|
|
| 55 |
"gpu"
|
| 56 |
],
|
| 57 |
"default_backend": "cpu",
|
| 58 |
-
"min_runtime_version": "0.
|
| 59 |
"recommended": [
|
| 60 |
{
|
| 61 |
"platform": "macos",
|
|
@@ -140,9 +150,11 @@
|
|
| 140 |
}
|
| 141 |
],
|
| 142 |
"known_issues": [
|
|
|
|
| 143 |
"Audio encoder does not run on the GPU delegates: on Metal/WebGPU the 30 s window dispatches 90001+ workgroups per stem kernel (limit 65535); on Adreno (OpenCL, all 1857 nodes delegated) the fp16 path returns an empty transcript; keep audio_backend=cpu",
|
| 144 |
"Audio placeholder tokens <|object_ref_start|>/<|object_ref_end|>/<|box_start|> are added to the bundled tokenizer.json (ids 151646/151647/151648, the ids the model was trained with); the upstream tokenizer.json has no string for them"
|
| 145 |
-
]
|
|
|
|
| 146 |
}
|
| 147 |
]
|
| 148 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"manifest_schema": "0.1.3",
|
| 3 |
"repo": "litert-community/VibeVoice-ASR-BitNet",
|
| 4 |
+
"generated": "2026-11-09",
|
| 5 |
"generator": "make_manifest.py",
|
| 6 |
"model": {
|
| 7 |
"display_name": "VibeVoice-ASR-BitNet (speech-to-text LLM, 24 kHz audio in)",
|
|
|
|
| 22 |
"variants": [
|
| 23 |
{
|
| 24 |
"file": "VibeVoice-ASR-BitNet.litertlm",
|
| 25 |
+
"sha256": "498ad85c609fb3821989db5f3767a07e38b82d5554bf26f73e091d939d9c1b0b",
|
| 26 |
+
"size_bytes": 1983053824,
|
| 27 |
"sections": [
|
| 28 |
{
|
| 29 |
"type": "LlmMetadataProto",
|
|
|
|
| 35 |
},
|
| 36 |
{
|
| 37 |
"type": "TFLiteModel",
|
| 38 |
+
"size_bytes": 3650664,
|
| 39 |
"model_type": "tf_lite_embedder"
|
| 40 |
},
|
| 41 |
{
|
| 42 |
"type": "TFLiteModel",
|
| 43 |
+
"size_bytes": 20937360,
|
| 44 |
"model_type": "tf_lite_prefill_decode"
|
| 45 |
},
|
| 46 |
{
|
| 47 |
"type": "TFLiteModel",
|
| 48 |
"size_bytes": 951122160,
|
| 49 |
"model_type": "tf_lite_audio_encoder_hw"
|
| 50 |
+
},
|
| 51 |
+
{
|
| 52 |
+
"type": "TFLiteWeights",
|
| 53 |
+
"size_bytes": 233373696,
|
| 54 |
+
"model_type": "tf_lite_embedder"
|
| 55 |
+
},
|
| 56 |
+
{
|
| 57 |
+
"type": "TFLiteWeights",
|
| 58 |
+
"size_bytes": 771784704,
|
| 59 |
+
"model_type": "tf_lite_prefill_decode"
|
| 60 |
}
|
| 61 |
],
|
| 62 |
"quantization": "LM: per-tensor ternary weights (VibeASR.cpp absmean, applied at conversion) stored as int4 blockwise-128 min-max (exact for a ternary tensor), int8 embedding; audio encoder: int8 dynamic on linears, fp32 convs, 30 s (225-frame) window",
|
|
|
|
| 65 |
"gpu"
|
| 66 |
],
|
| 67 |
"default_backend": "cpu",
|
| 68 |
+
"min_runtime_version": "0.19.0",
|
| 69 |
"recommended": [
|
| 70 |
{
|
| 71 |
"platform": "macos",
|
|
|
|
| 150 |
}
|
| 151 |
],
|
| 152 |
"known_issues": [
|
| 153 |
+
"Android GPU needs LiteRT-LM 0.19.0 or later: since 2026-11-09 this file's weights sit in TFLiteWeights sections at the end of the file (LiteRT's external weights), and the GPU engine of the v0.18.0 and v0.18.1 Android builds crashes (SIGSEGV) on such a file (https://github.com/google-ai-edge/LiteRT-LM/issues/3986); the earlier file with the weights inside the graph stays at revision 193b2d579a786206286ac36e74b6ea513562bdf5 (https://huggingface.co/litert-community/VibeVoice-ASR-BitNet/tree/193b2d579a786206286ac36e74b6ea513562bdf5)",
|
| 154 |
"Audio encoder does not run on the GPU delegates: on Metal/WebGPU the 30 s window dispatches 90001+ workgroups per stem kernel (limit 65535); on Adreno (OpenCL, all 1857 nodes delegated) the fp16 path returns an empty transcript; keep audio_backend=cpu",
|
| 155 |
"Audio placeholder tokens <|object_ref_start|>/<|object_ref_end|>/<|box_start|> are added to the bundled tokenizer.json (ids 151646/151647/151648, the ids the model was trained with); the upstream tokenizer.json has no string for them"
|
| 156 |
+
],
|
| 157 |
+
"context_length": 2048
|
| 158 |
}
|
| 159 |
]
|
| 160 |
}
|