Instructions to use litert-community/Rumble-2-Flash-LiteRT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT-LM
How to use litert-community/Rumble-2-Flash-LiteRT with LiteRT-LM:
# LiteRT-LM runs on various platforms (Android, iOS, Windows, Linux, macOS, IoT, Web/WASM) # and supports many APIs (C++, Python, Kotlin, Swift, JavaScript, Flutter). # For platform-specific integration guides, please refer to the official developer website: # https://ai.google.dev/edge/litert-lm # To try LiteRT-LM, the easiest way is to use our CLI tool. # 1. Install the LiteRT-LM CLI tool: pip install -U litert-lm # 2. Download and run this model locally: # See: https://ai.google.dev/edge/litert-lm/cli # A single .litertlm file in the repo is picked automatically; otherwise the CLI asks which one to run # (or pass its name right after the repo id). litert-lm run \ --from-huggingface-repo=litert-community/Rumble-2-Flash-LiteRT \ --prompt="Write me a poem"
- LiteRT
How to use litert-community/Rumble-2-Flash-LiteRT with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Rumble-2-Flash for LiteRT-LM
On-device speech-to-text in a single file.
This is Reson8/Rumble-2-Flash
(a 0.5B Qwen3-ASR model covering 45 languages) packaged as a self-contained
.litertlm bundle for
LiteRT-LM Omni.
The file contains the INT8 audio encoder and projector, the 14-layer Qwen3
decoder with stateful KV cache, the token embedder, the Hugging Face tokenizer,
and LlmMetadata / ExecutorMetadata. No Python and no network are needed at
inference time.
- Architecture: 128-bin log-mel front-end, 18-layer audio encoder
(
tf_lite_audio_encoder_hw), 14-layer Qwen3 decoder (tf_lite_prefill_decode+tf_lite_embedder), 99,630-token vocabulary - Streaming: 5-second sliding windows, merged across overlaps by the LiteRT-LM ASR session
- Transcription style (fixed at export): numbers as digits (
<digits>), fillers dropped (<nofiller>), repeated words kept (<verbatim>), language auto-detected. These are Rumble's defaults.
Files
| File | Size | Backend | Notes |
|---|---|---|---|
rumble_2_flash_5s_i8.litertlm |
596 MB | CPU (GPU untested) | INT8 dynamic-range weights, 5 s window |
rumble_asr_extension.py |
5 KB | — | litert-torch extension used for the export |
omni_runner_skip_until.patch |
4 KB | — | Adds --asr_decode_skip_until_token_id to omni_runner |
One runtime override is required. Rumble-2-Flash remaps the Qwen3 vocabulary, so its
<asr_text>token is 99623, while LiteRT-LM'sqwen3-asrruntime defaultsdecode_skip_until_token_idto the stock Qwen3-ASR value 151704. Without the override every transcript is empty. The value cannot be embedded in the file: a.litertlmholds a single top-level capability section, and this model usesLlmMetadata.
omni_runner: pass--model_name=qwen3-asr --asr_decode_skip_until_token_id=99623(requiresomni_runner_skip_until.patch).- C++ API: pass a
Speech2TextMetadataas the fourth argument ofOmniEngine::Create(see below).
Usage with LiteRT-LM Omni
1. Download
pip install -U huggingface_hub
hf download litert-community/Rumble-2-Flash-LiteRT --local-dir rumble
2. Build omni_runner with the patch
Tested against LiteRT-LM commit
7ee9bdb.
Prerequisites (Bazel, Android NDK r28b+) are in the LiteRT-LM
build guide.
git clone https://github.com/google-ai-edge/LiteRT-LM && cd LiteRT-LM
git apply ../rumble/omni_runner_skip_until.patch
# Desktop (macOS / Linux)
bazel build -c opt //omni:omni_runner
# Android arm64
export ANDROID_NDK_HOME=/path/to/android-ndk-r28c
bazel build -c opt --config=android_arm64 //omni:omni_runner
3. Run
bazel-bin/omni/omni_runner \
--model_name=qwen3-asr \
--model_path=../rumble/rumble_2_flash_5s_i8.litertlm \
--audio_path=/path/to/audio.wav \
--asr_decode_skip_until_token_id=99623
--model_name must be given explicitly because it cannot be inferred from the
file name. WAV input of any sample rate and channel count is resampled to
16 kHz mono.
Android (adb)
D=/data/local/tmp/omni
adb shell mkdir -p $D
adb push bazel-bin/omni/omni_runner prebuilt/android_arm64/*.so \
../rumble/rumble_2_flash_5s_i8.litertlm audio.wav $D
adb shell "cd $D && LD_LIBRARY_PATH=$D ./omni_runner --backend=cpu \
--cache_dir=$D/cache --model_name=qwen3-asr \
--model_path=$D/rumble_2_flash_5s_i8.litertlm --audio_path=$D/audio.wav \
--asr_decode_skip_until_token_id=99623"
Android (Termux, push-to-talk)
Copy omni_runner, the two prebuilt/android_arm64/*.so files and the
.litertlm into ~/rumble (binaries cannot execute from shared storage), and
grant Termux the microphone permission. Then:
pkg install ffmpeg termux-api
cat > ~/rumble/asr <<'EOF'
#!/data/data/com.termux/files/usr/bin/sh
D=$HOME/rumble
LD_LIBRARY_PATH=$D "$D/omni_runner" --backend="${BACKEND:-cpu}" --cache_dir="$D/cache" \
--model_name=qwen3-asr --model_path="$D/rumble_2_flash_5s_i8.litertlm" \
--audio_path="$1" --asr_decode_skip_until_token_id=99623 2>/dev/null
EOF
chmod +x ~/rumble/asr
termux-microphone-record -f ~/rec.m4a -l 10 && sleep 11
ffmpeg -y -loglevel error -i ~/rec.m4a ~/rec.wav && ~/rumble/asr ~/rec.wav
C++ API
#include "omni/omni_engine.h"
#include "runtime/proto/speech2text_metadata.pb.h"
#include "runtime/proto/speech2text_session_config.pb.h"
litert::omni::OmniEngine::Settings settings{
.backend = litert::omni::OmniEngine::Settings::Backend::kCpu,
.cache_dir = "/data/local/tmp/omni_cache",
.model_path = "/path/to/rumble_2_flash_5s_i8.litertlm",
};
litert::lm::proto::Speech2TextSessionConfig session_config;
litert::lm::proto::Speech2TextMetadata metadata;
metadata.set_decode_skip_until_token_id(99623); // Rumble's <asr_text>
ABSL_ASSIGN_OR_RETURN(
auto engine, litert::omni::OmniEngine::Create(
"qwen3-asr", settings, &session_config, &metadata));
Link //omni:omni_engine, //omni:omni_session and
//omni/speech2text:qwen3_asr. The rest of the session API is unchanged from
the LiteRT-LM Omni README.
Validation
Sample: runtime/components/testdata/audio_sample.wav from LiteRT-LM (5.8 s,
English).
| Runtime | Transcript |
|---|---|
| Hugging Face Transformers (FP32, reference) | What paintings were inside the dead person's old kingdom tombs? |
.litertlm, macOS CPU |
What paintings were inside the dead person's old Kingdom tombs. |
.litertlm, Android CPU (Snapdragon 8 Elite Gen 5) |
What paintings were inside the dead person's old Kingdom tombs. |
- Audio encoder vs. PyTorch: FP32 max abs diff 6.7e-7; INT8 minimum per-token cosine similarity 0.986.
- Android CPU (Snapdragon 8 Elite Gen 5, Termux): 5.8 s clip transcribed in 1.5 s, including first-run cache creation.
Not yet validated: GPU backend, NPU, long-form audio across many windows, and non-English accuracy.
How it was converted
The built-in qwen3_asr extension in
litert-torch hardcodes stock
Qwen3 prompt token IDs (for example 151644), which are outside Rumble's
vocabulary. rumble_asr_extension.py rebuilds the encoder prompt from the
checkpoint's own chat template and tokenizer, and takes the stop token from it.
pip install -r litert-torch/requirements.txt && pip install -e ./litert-torch
python -m litert_torch.generative.export_hf.export_main \
--model=Reson8/Rumble-2-Flash \
--output_dir=out \
--extension_modules=./rumble_asr_extension.py \
--quantization_recipe=dynamic_wi8_afp32
Exported with litert-torch commit
9d531ea,
transformers 5.23.0 and torch 2.14.0.
License
Apache 2.0, inherited from Reson8/Rumble-2-Flash. All credit for the model goes to Reson8; this repository only repackages it for LiteRT-LM.
- Downloads last month
- 43