Rumble-2-Flash-8bit / README.md
TheEvil1's picture
Add Rumble-2-Flash converted to MLX with an 8-bit decoder
feb7feb verified
|
Raw History Blame Contribute Delete
3.2 kB
metadata
library_name: mlx-audio
pipeline_tag: automatic-speech-recognition
license: apache-2.0
base_model: Reson8/Rumble-2-Flash
base_model_relation: quantized
language:
  - en
  - de
  - fr
  - es
  - it
  - pt
  - nl
  - ru
  - zh
  - ja
  - ko
  - ar
  - hi
  - tr
  - pl
  - sv
  - da
  - fi
  - cs
  - el
  - ro
  - hu
  - id
  - vi
  - th
  - ms
  - fa
  - mk
tags:
  - mlx
  - speech-to-text
  - qwen3-asr

Rumble-2-Flash (MLX, 8-bit)

Reson8 Rumble-2-Flash for Apple Silicon. Rumble-2-Flash is a 0.5B speech recognition model in the Qwen3-ASR architecture, with a 14-layer text decoder and a 99,630-token vocabulary.

The text decoder is quantized to 8 bits (affine, group size 64). The audio encoder and the projector stay in BF16. The weights take 692 MB.

Conversion

Converted from the BF16 checkpoint at revision a0c34287229103b706c670d70791683e34379732 with convert-rumble-2-flash.py and mlx-audio 0.6.3:

  • The transformers 5 weight names are mapped to the original Qwen3-ASR layout that mlx-audio loads: model.audio_tower.* to audio_tower.*, model.multi_modal_projector.linear_1/2 to audio_tower.proj1/proj2 and model.language_model.* to model.*. The convolution kernels move to MLX's channel-last order.
  • config.json also carries text_config.rope_theta, which transformers 5 keeps in rope_parameters.
  • Tokenizer, chat template and processor files are copied unchanged. The optional CTC head of the original repository is left out.

Usage

Rumble-2-Flash numbers <|im_end|> and <|endoftext|> differently from Qwen3-ASR (99564 and 99562). mlx-audio 0.6.3 reads them from the tokenizer. mlx-audio-swift needs a fix for this; without it, generation does not stop at the end of the transcript.

The model was trained with three control tokens at the start of the system prompt. Its chat template's defaults are <digits><nofiller><verbatim>: numbers as digits, no filler words, repeated words kept.

from mlx_audio.stt import load

model = load("TypeWhisper/Rumble-2-Flash-8bit")
result = model.generate(
    "audio.wav",
    language="German",  # leave out to detect the language
    system_prompt="<digits><nofiller><verbatim>",
)
print(result.text)

Accuracy

Word error rate on the TypeWhisper benchmark, mean over the suites both models ran (22 suites in English, German, French, Spanish, Italian, Portuguese and Russian), measured with mlx-audio on a Mac mini:

Model WER Real-time factor
Rumble-2-Flash 8-bit (this repository) 6.3 % 0.013
Qwen3-ASR 0.6B 8-bit 10.2 % 0.035

The converted model matches Reson8's hosted Rumble-2-Flash API on the 25 suites both were run on.

This checkpoint is a model choice in the Qwen3 ASR plugin of TypeWhisper.

License

Rumble-2-Flash is licensed under Apache 2.0 by Reson8. The license is included in this repository.