Instructions to use kaptionai/Rumble-2-Flash-GGUF-wasm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use kaptionai/Rumble-2-Flash-GGUF-wasm with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0 # Run inference directly in the terminal: llama cli -hf kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0 # Run inference directly in the terminal: llama cli -hf kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0
Use Docker
docker model run hf.co/kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use kaptionai/Rumble-2-Flash-GGUF-wasm with Ollama:
ollama run hf.co/kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use kaptionai/Rumble-2-Flash-GGUF-wasm with Docker Model Runner:
docker model run hf.co/kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0
- Lemonade
How to use kaptionai/Rumble-2-Flash-GGUF-wasm with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull kaptionai/Rumble-2-Flash-GGUF-wasm:Q8_0
Run and chat with the model
lemonade run user.Rumble-2-Flash-GGUF-wasm-Q8_0
List all available models
lemonade list
- Atomic Chat
Rumble-2-Flash-GGUF-wasm (Kaption)
Two GGUF files to run Reson8/Rumble-2-Flash on the CPU in a browser, with a WebAssembly build of llama.cpp + libmtmd. 550 MB in total. They are chosen for WebAssembly speed: int8 (Q8_0) weights have the fastest wasm SIMD kernels. The layout is the same as kaptionai/Qwen3-ASR-0.6B-GGUF-wasm.
| File | Bytes | SHA-256 | Origin |
|---|---|---|---|
Rumble-2-Flash.Q8_0.gguf |
346,355,584 | 600821d04cd1a7caa42f4622f6ac42b84e6db00f73196ed73493d06c221886bc |
Unmodified copy of AtomicChat's Rumble-2-Flash-Q8_0.gguf, renamed |
Rumble-2-Flash.mmproj-Q8_0-convq8.gguf |
204,053,792 | cd4772d6206c34adf1788d1a65d3693734dc28874aed966b7a69be7c3b005f75 |
Kaption conversion of AtomicChat's mmproj-Rumble-2-Flash-Q8_0.gguf (see below) |
retype-mmproj.py |
2,900 | 33e355c389607a80b9e6d288f174ea6611ada7fa17942b03d65b0a0b614ec5b5 |
Kaption's script that makes the projector file |
LICENSE |
11,358 | cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 |
Apache License 2.0 text |
- Base model: Reson8/Rumble-2-Flash, Apache-2.0, revision
a0c34287229103b706c670d70791683e34379732. It is a fine-tune of Qwen/Qwen3-ASR-0.6B (Apache-2.0) with the decoder cut to 14 layers and the vocabulary cut to 99,630 tokens. - GGUF conversion and quantisation: AtomicChat/Rumble-2-Flash-GGUF,
revision
d036f2af3cd5229375714d7f1749792cce3c6a88. Their card states the source as Reson8/Rumble-2-Flash at92ec9141; the weights file is the same at both revisions. - License: Apache License 2.0, as declared by the base model and by AtomicChat. The text is included as
LICENSE. - Not endorsed by Reson8, AtomicChat or Alibaba Qwen. All credit for the model belongs to its authors.
What was changed in the audio projector
Only three tensors, all from AtomicChat's own values; every other tensor and all metadata are copied unchanged:
a.conv2d.2.weightanda.conv2d.3.weight(3x3 convolutions, 480 -> 480 channels): the F16 values, reshaped from 4-D[3, 3, 480, 480]to a 2-D[4320, 480]matrix (row-major, the same element order as im2col's columns) and quantised to Q8_0.a.conv_out.weight([7680, 896]): the F16 values quantised to Q8_0.
retype-mmproj.py reproduces the file exactly (checked: same SHA-256):
python retype-mmproj.py <llama.cpp>/gguf-py mmproj-Rumble-2-Flash-Q8_0.gguf out.gguf --mat q8_0 --conv q8_0.
Runtime
A WebAssembly build of llama.cpp + libmtmd that recognises the 2-D convolution weights and runs them as im2col plus an int8 matrix multiply (Kaption's llama.cpp patch). Stock llama.cpp expects 4-D weights and will not run this projector: use AtomicChat's files with stock llama.cpp. The decoder file runs anywhere AtomicChat's does.
The prompt needs Reson8's control tokens <digits><nofiller><verbatim> in the system turn (the defaults of the chat
template embedded in the GGUF), one audio turn and the assistant turn. Decoding is greedy.
Known limits (measured by Kaption)
- Chinese is weaker than with Qwen3-ASR-0.6B. In this runtime, on 15 FLEURS clips: character error rate 7.4 against 6.6, which is inside the noise of 15 clips; on Kaption's larger native test (40 clips) it was 7.4 against 5.4. English, Spanish and Portuguese are better (FLEURS, 48 clips: word error rate 4.30 against 6.35).
- It invents text on digital silence. On 36 synthetic clips without speech it wrote looped sentences on 10 (up to 28 repeats of one sentence). Put a voice-activity gate in front and do not send silence to the model.
- Short clips need the padding fix. llama.cpp's Qwen3-ASR audio path passes the zero padding at the end of a clip to the model as sound, and clips of 1 to 3 s then end with words that were never said (word error rate 56.8 without the fix, 25.1 with it, 106 clips). AtomicChat's card describes the fix.
- No Hebrew. The model's 45 languages are the ones listed in the metadata above.
- No word timestamps. The output is text only.
- Tested in Chrome and Edge. Other browsers were not tested.
- Downloads last month
- 98
8-bit