Rumble-2-Flash-GGUF-wasm (Kaption)

Two GGUF files to run Reson8/Rumble-2-Flash on the CPU in a browser, with a WebAssembly build of llama.cpp + libmtmd. 550 MB in total. They are chosen for WebAssembly speed: int8 (Q8_0) weights have the fastest wasm SIMD kernels. The layout is the same as kaptionai/Qwen3-ASR-0.6B-GGUF-wasm.

File Bytes SHA-256 Origin
Rumble-2-Flash.Q8_0.gguf 346,355,584 600821d04cd1a7caa42f4622f6ac42b84e6db00f73196ed73493d06c221886bc Unmodified copy of AtomicChat's Rumble-2-Flash-Q8_0.gguf, renamed
Rumble-2-Flash.mmproj-Q8_0-convq8.gguf 204,053,792 cd4772d6206c34adf1788d1a65d3693734dc28874aed966b7a69be7c3b005f75 Kaption conversion of AtomicChat's mmproj-Rumble-2-Flash-Q8_0.gguf (see below)
retype-mmproj.py 2,900 33e355c389607a80b9e6d288f174ea6611ada7fa17942b03d65b0a0b614ec5b5 Kaption's script that makes the projector file
LICENSE 11,358 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 Apache License 2.0 text
  • Base model: Reson8/Rumble-2-Flash, Apache-2.0, revision a0c34287229103b706c670d70791683e34379732. It is a fine-tune of Qwen/Qwen3-ASR-0.6B (Apache-2.0) with the decoder cut to 14 layers and the vocabulary cut to 99,630 tokens.
  • GGUF conversion and quantisation: AtomicChat/Rumble-2-Flash-GGUF, revision d036f2af3cd5229375714d7f1749792cce3c6a88. Their card states the source as Reson8/Rumble-2-Flash at 92ec9141; the weights file is the same at both revisions.
  • License: Apache License 2.0, as declared by the base model and by AtomicChat. The text is included as LICENSE.
  • Not endorsed by Reson8, AtomicChat or Alibaba Qwen. All credit for the model belongs to its authors.

What was changed in the audio projector

Only three tensors, all from AtomicChat's own values; every other tensor and all metadata are copied unchanged:

  • a.conv2d.2.weight and a.conv2d.3.weight (3x3 convolutions, 480 -> 480 channels): the F16 values, reshaped from 4-D [3, 3, 480, 480] to a 2-D [4320, 480] matrix (row-major, the same element order as im2col's columns) and quantised to Q8_0.
  • a.conv_out.weight ([7680, 896]): the F16 values quantised to Q8_0.

retype-mmproj.py reproduces the file exactly (checked: same SHA-256): python retype-mmproj.py <llama.cpp>/gguf-py mmproj-Rumble-2-Flash-Q8_0.gguf out.gguf --mat q8_0 --conv q8_0.

Runtime

A WebAssembly build of llama.cpp + libmtmd that recognises the 2-D convolution weights and runs them as im2col plus an int8 matrix multiply (Kaption's llama.cpp patch). Stock llama.cpp expects 4-D weights and will not run this projector: use AtomicChat's files with stock llama.cpp. The decoder file runs anywhere AtomicChat's does.

The prompt needs Reson8's control tokens <digits><nofiller><verbatim> in the system turn (the defaults of the chat template embedded in the GGUF), one audio turn and the assistant turn. Decoding is greedy.

Known limits (measured by Kaption)

  • Chinese is weaker than with Qwen3-ASR-0.6B. In this runtime, on 15 FLEURS clips: character error rate 7.4 against 6.6, which is inside the noise of 15 clips; on Kaption's larger native test (40 clips) it was 7.4 against 5.4. English, Spanish and Portuguese are better (FLEURS, 48 clips: word error rate 4.30 against 6.35).
  • It invents text on digital silence. On 36 synthetic clips without speech it wrote looped sentences on 10 (up to 28 repeats of one sentence). Put a voice-activity gate in front and do not send silence to the model.
  • Short clips need the padding fix. llama.cpp's Qwen3-ASR audio path passes the zero padding at the end of a clip to the model as sound, and clips of 1 to 3 s then end with words that were never said (word error rate 56.8 without the fix, 25.1 with it, 106 clips). AtomicChat's card describes the fix.
  • No Hebrew. The model's 45 languages are the ones listed in the metadata above.
  • No word timestamps. The output is text only.
  • Tested in Chrome and Edge. Other browsers were not tested.
Downloads last month
98
GGUF
Model size
0.3B params
Architecture
qwen3vl
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for kaptionai/Rumble-2-Flash-GGUF-wasm

Quantized
(10)
this model