rumble-2-flash-web / README.md
kaptionai's picture
Rumble-2-Flash: WebGPU package (int8 encoder, 4-bit decoder)
5b3dde6 verified
|
Raw History Blame Contribute Delete
5.4 kB
---
license: apache-2.0
base_model: Reson8/Rumble-2-Flash
pipeline_tag: automatic-speech-recognition
library_name: onnxruntime
language: [af, ar, bn, bs, ca, ceb, cs, da, de, el, en, es, eu, fa, fi, fil, fr, hi, hr, hu, id, is, it, ja, ko, lt, lv, mk, ms, mt, nb, nl, pl, pt, ro, ru, sq, sr, sv, th, tr, uk, vi, yue, zh]
tags:
- onnx
- onnxruntime-web
- webgpu
- qwen3-asr
- kaption
---
# Rumble-2-Flash for the browser (WebGPU), converted by Kaption
ONNX files to run [Reson8/Rumble-2-Flash](https://huggingface.co/Reson8/Rumble-2-Flash) in a browser with
onnxruntime-web on WebGPU. 494 MB in total. The layout is the same as
[kaptionai/qwen3-asr-web](https://huggingface.co/kaptionai/qwen3-asr-web).
- **Base model:** Reson8/Rumble-2-Flash, Apache-2.0, revision `a0c34287229103b706c670d70791683e34379732`. It is a
fine-tune of [Qwen/Qwen3-ASR-0.6B](https://huggingface.co/Qwen/Qwen3-ASR-0.6B) (Apache-2.0) with the decoder cut to
14 layers and the vocabulary cut to 99,630 tokens.
- **GGUF quantisation of the same model:** [AtomicChat/Rumble-2-Flash-GGUF](https://huggingface.co/AtomicChat/Rumble-2-Flash-GGUF).
No file here comes from that repository; their model card is where the relation to Qwen3-ASR-0.6B is documented.
- **These files:** all converted by Kaption from the base model's `model.safetensors`, except `tokenizer.json`
(Reson8's file, unchanged).
- Not endorsed by Reson8, AtomicChat or Alibaba Qwen. All credit for the model belongs to its authors.
## Files
| File | Bytes | SHA-256 | Origin |
|---|---:|---|---|
| `encoder.int8f16.onnx` | 202,340,869 | `f6fb2942f0d8048967f9d60b76d531642d18e44e6e431c289aaaa7ad9d51339b` | Kaption conversion |
| `decoder.gqa-q4f16.onnx` | 182,951 | `07d102f48cb4ed993cb78c3112ec0f49b4423aab167fb96e26026bca13ae3525` | Kaption conversion |
| `decoder_weights.q4f16.data` | 181,774,208 | `b1cc416f9ec33741e2f4b35bb6f7bd4ef07de92cd2828534d92c6184d4b20b07` | Kaption conversion (external data of the decoder graph) |
| `embed_tokens.int8.bin` | 102,021,120 | `be3ebc783a5b42edda8de10fce666b10960eb2ae25e42ff6565399aa46d17953` | Kaption conversion |
| `embed_scales.f32.bin` | 398,520 | `b192623ed103834f0ada43b5ce2a47bb84b6b0bfda979dc9fe63cd90c57f082e` | Kaption conversion |
| `tokenizer.json` | 7,422,837 | `60d63d6d615abcf61470a29a74b8c7434a96f87f2e97bca710cb767e076c0040` | Reson8/Rumble-2-Flash, unchanged |
| `mel_filters.json` | 132,427 | `dcdaf139353616855b4f85e304c075f5f3442573d40637ae0f031db705626181` | The same file as in kaptionai/qwen3-asr-web (128 mel bins, 16 kHz) |
| `prompt_config.json` | 1,528 | `df4482b2b51d485481bbac1fcd3469e06bc6f07492e8a77e61e478f0d79b6ac1` | Written by Kaption: prompt token ids under the pruned vocabulary |
| `LICENSE` | 11,358 | `cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30` | Apache License 2.0 text |
## How the files were made
- `encoder.int8f16.onnx`: the audio encoder and projector exported to ONNX (18 layers, checked against the
Transformers model), every MatMul weight quantised to ONNX Runtime `MatMulNBits` (8-bit, block 32), then converted to
fp16 activations inside with fp32 inputs and outputs. It takes a batch of 8 s windows: `mel` [W, 128, 800] and
`valid` [W, 104] in, `emb` [W, 104, 1024] out.
- `decoder.gqa-q4f16.onnx` + `decoder_weights.q4f16.data`: the 14-layer text decoder built with onnxruntime-genai's
model builder (4-bit weights, WebGPU target, embeddings left out), then rewritten as one graph for the prefill and
every step: `GroupQueryAttention` with rotary and q/k norm inside, only the last position reaches the output head, and
the greedy pick runs on the GPU, so `next_token` is the only small output. Float inputs and outputs are fp16.
- `embed_tokens.int8.bin` + `embed_scales.f32.bin`: the token embedding table (99,630 rows of 1,024), int8 per row with
one fp32 scale per row (largest error 0.001). The embedding lookup is done outside the graph.
- `prompt_config.json`: token ids of the fixed prompt. The system turn carries Reson8's control tokens
`<digits><nofiller><verbatim>` (the defaults of the model's chat template), then one audio turn, then the assistant
turn. It also holds the `language <Name>` ids used to force a language.
## Runtime
onnxruntime-web with the WebGPU execution provider. The GPU must support `shader-f16`. The decoder graph does not run
on ONNX Runtime's CPU provider. Audio is 16 kHz mono, log-mel with 128 bins, cut into 8 s encoder windows; decoding is
greedy.
## Known limits (measured by Kaption)
- **Chinese is weaker than with Qwen3-ASR-0.6B.** On 15 FLEURS clips: character error rate 8.3 against 5.3 for the
Qwen3-ASR WebGPU package. Spanish and Portuguese are better (FLEURS, 240 clips in en/es/pt: word error rate 4.08
against 6.50).
- **It invents text on digital silence.** On 36 synthetic clips without speech it wrote text on 4, each time one
sentence repeated about 10 times. Put a voice-activity gate in front and do not send silence to the model.
- **No Hebrew.** The model's 45 languages are the ones listed in the metadata above.
- **No word timestamps.** The output is text only.
- Tested in Chrome on Apple silicon and on an Intel UHD 620 laptop. Other browsers were not tested.
## License
Apache-2.0 (`LICENSE`), the same as Reson8/Rumble-2-Flash and Qwen/Qwen3-ASR-0.6B. Copyright of the model weights
stays with their authors.