File size: 5,396 Bytes
5b3dde6
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
---
license: apache-2.0
base_model: Reson8/Rumble-2-Flash
pipeline_tag: automatic-speech-recognition
library_name: onnxruntime
language: [af, ar, bn, bs, ca, ceb, cs, da, de, el, en, es, eu, fa, fi, fil, fr, hi, hr, hu, id, is, it, ja, ko, lt, lv, mk, ms, mt, nb, nl, pl, pt, ro, ru, sq, sr, sv, th, tr, uk, vi, yue, zh]
tags:
  - onnx
  - onnxruntime-web
  - webgpu
  - qwen3-asr
  - kaption
---

# Rumble-2-Flash for the browser (WebGPU), converted by Kaption

ONNX files to run [Reson8/Rumble-2-Flash](https://huggingface.co/Reson8/Rumble-2-Flash) in a browser with
onnxruntime-web on WebGPU. 494 MB in total. The layout is the same as
[kaptionai/qwen3-asr-web](https://huggingface.co/kaptionai/qwen3-asr-web).

- **Base model:** Reson8/Rumble-2-Flash, Apache-2.0, revision `a0c34287229103b706c670d70791683e34379732`. It is a
  fine-tune of [Qwen/Qwen3-ASR-0.6B](https://huggingface.co/Qwen/Qwen3-ASR-0.6B) (Apache-2.0) with the decoder cut to
  14 layers and the vocabulary cut to 99,630 tokens.
- **GGUF quantisation of the same model:** [AtomicChat/Rumble-2-Flash-GGUF](https://huggingface.co/AtomicChat/Rumble-2-Flash-GGUF).
  No file here comes from that repository; their model card is where the relation to Qwen3-ASR-0.6B is documented.
- **These files:** all converted by Kaption from the base model's `model.safetensors`, except `tokenizer.json`
  (Reson8's file, unchanged).
- Not endorsed by Reson8, AtomicChat or Alibaba Qwen. All credit for the model belongs to its authors.

## Files

| File | Bytes | SHA-256 | Origin |
|---|---:|---|---|
| `encoder.int8f16.onnx` | 202,340,869 | `f6fb2942f0d8048967f9d60b76d531642d18e44e6e431c289aaaa7ad9d51339b` | Kaption conversion |
| `decoder.gqa-q4f16.onnx` | 182,951 | `07d102f48cb4ed993cb78c3112ec0f49b4423aab167fb96e26026bca13ae3525` | Kaption conversion |
| `decoder_weights.q4f16.data` | 181,774,208 | `b1cc416f9ec33741e2f4b35bb6f7bd4ef07de92cd2828534d92c6184d4b20b07` | Kaption conversion (external data of the decoder graph) |
| `embed_tokens.int8.bin` | 102,021,120 | `be3ebc783a5b42edda8de10fce666b10960eb2ae25e42ff6565399aa46d17953` | Kaption conversion |
| `embed_scales.f32.bin` | 398,520 | `b192623ed103834f0ada43b5ce2a47bb84b6b0bfda979dc9fe63cd90c57f082e` | Kaption conversion |
| `tokenizer.json` | 7,422,837 | `60d63d6d615abcf61470a29a74b8c7434a96f87f2e97bca710cb767e076c0040` | Reson8/Rumble-2-Flash, unchanged |
| `mel_filters.json` | 132,427 | `dcdaf139353616855b4f85e304c075f5f3442573d40637ae0f031db705626181` | The same file as in kaptionai/qwen3-asr-web (128 mel bins, 16 kHz) |
| `prompt_config.json` | 1,528 | `df4482b2b51d485481bbac1fcd3469e06bc6f07492e8a77e61e478f0d79b6ac1` | Written by Kaption: prompt token ids under the pruned vocabulary |
| `LICENSE` | 11,358 | `cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30` | Apache License 2.0 text |

## How the files were made

- `encoder.int8f16.onnx`: the audio encoder and projector exported to ONNX (18 layers, checked against the
  Transformers model), every MatMul weight quantised to ONNX Runtime `MatMulNBits` (8-bit, block 32), then converted to
  fp16 activations inside with fp32 inputs and outputs. It takes a batch of 8 s windows: `mel` [W, 128, 800] and
  `valid` [W, 104] in, `emb` [W, 104, 1024] out.
- `decoder.gqa-q4f16.onnx` + `decoder_weights.q4f16.data`: the 14-layer text decoder built with onnxruntime-genai's
  model builder (4-bit weights, WebGPU target, embeddings left out), then rewritten as one graph for the prefill and
  every step: `GroupQueryAttention` with rotary and q/k norm inside, only the last position reaches the output head, and
  the greedy pick runs on the GPU, so `next_token` is the only small output. Float inputs and outputs are fp16.
- `embed_tokens.int8.bin` + `embed_scales.f32.bin`: the token embedding table (99,630 rows of 1,024), int8 per row with
  one fp32 scale per row (largest error 0.001). The embedding lookup is done outside the graph.
- `prompt_config.json`: token ids of the fixed prompt. The system turn carries Reson8's control tokens
  `<digits><nofiller><verbatim>` (the defaults of the model's chat template), then one audio turn, then the assistant
  turn. It also holds the `language <Name>` ids used to force a language.

## Runtime

onnxruntime-web with the WebGPU execution provider. The GPU must support `shader-f16`. The decoder graph does not run
on ONNX Runtime's CPU provider. Audio is 16 kHz mono, log-mel with 128 bins, cut into 8 s encoder windows; decoding is
greedy.

## Known limits (measured by Kaption)

- **Chinese is weaker than with Qwen3-ASR-0.6B.** On 15 FLEURS clips: character error rate 8.3 against 5.3 for the
  Qwen3-ASR WebGPU package. Spanish and Portuguese are better (FLEURS, 240 clips in en/es/pt: word error rate 4.08
  against 6.50).
- **It invents text on digital silence.** On 36 synthetic clips without speech it wrote text on 4, each time one
  sentence repeated about 10 times. Put a voice-activity gate in front and do not send silence to the model.
- **No Hebrew.** The model's 45 languages are the ones listed in the metadata above.
- **No word timestamps.** The output is text only.
- Tested in Chrome on Apple silicon and on an Intel UHD 620 laptop. Other browsers were not tested.

## License

Apache-2.0 (`LICENSE`), the same as Reson8/Rumble-2-Flash and Qwen/Qwen3-ASR-0.6B. Copyright of the model weights
stays with their authors.