Instructions to use TypeWhisper/Rumble-2-Flash-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use TypeWhisper/Rumble-2-Flash-8bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download TypeWhisper/Rumble-2-Flash-8bit --local-dir Rumble-2-Flash-8bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Rumble-2-Flash (MLX, 8-bit)
Reson8 Rumble-2-Flash for Apple Silicon. Rumble-2-Flash is a 0.5B speech recognition model in the Qwen3-ASR architecture, with a 14-layer text decoder and a 99,630-token vocabulary.
The text decoder is quantized to 8 bits (affine, group size 64). The audio encoder and the projector stay in BF16. The weights take 692 MB.
Conversion
Converted from the BF16 checkpoint at revision a0c34287229103b706c670d70791683e34379732
with convert-rumble-2-flash.py
and mlx-audio 0.6.3:
- The transformers 5 weight names are mapped to the original Qwen3-ASR layout that
mlx-audio loads:
model.audio_tower.*toaudio_tower.*,model.multi_modal_projector.linear_1/2toaudio_tower.proj1/proj2andmodel.language_model.*tomodel.*. The convolution kernels move to MLX's channel-last order. config.jsonalso carriestext_config.rope_theta, which transformers 5 keeps inrope_parameters.- Tokenizer, chat template and processor files are copied unchanged. The optional CTC head of the original repository is left out.
Usage
Rumble-2-Flash numbers <|im_end|> and <|endoftext|> differently from Qwen3-ASR (99564 and
99562). mlx-audio 0.6.3 reads them from the tokenizer. mlx-audio-swift needs
a fix
for this; without it, generation does not stop at the end of the transcript.
The model was trained with three control tokens at the start of the system prompt. Its chat
template's defaults are <digits><nofiller><verbatim>: numbers as digits, no filler words,
repeated words kept.
from mlx_audio.stt import load
model = load("TypeWhisper/Rumble-2-Flash-8bit")
result = model.generate(
"audio.wav",
language="German", # leave out to detect the language
system_prompt="<digits><nofiller><verbatim>",
)
print(result.text)
Accuracy
Word error rate on the TypeWhisper benchmark, mean over the suites both models ran (22 suites in English, German, French, Spanish, Italian, Portuguese and Russian), measured with mlx-audio on a Mac mini:
| Model | WER | Real-time factor |
|---|---|---|
| Rumble-2-Flash 8-bit (this repository) | 6.3 % | 0.013 |
| Qwen3-ASR 0.6B 8-bit | 10.2 % | 0.035 |
The converted model matches Reson8's hosted Rumble-2-Flash API on the 25 suites both were run on.
This checkpoint is a model choice in the Qwen3 ASR plugin of TypeWhisper.
License
Rumble-2-Flash is licensed under Apache 2.0 by Reson8. The license is included in this repository.
- Downloads last month
- 25
Quantized
Model tree for TypeWhisper/Rumble-2-Flash-8bit
Base model
Reson8/Rumble-2-Flash