Automatic Speech Recognition
GGUF
ggml
quantized
llama.cpp
qwen3-asr
speech-recognition
on-device
conversational
Instructions to use javierbits/Rumble-2-Flash-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use javierbits/Rumble-2-Flash-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf javierbits/Rumble-2-Flash-GGUF:Q5_K_M # Run inference directly in the terminal: llama cli -hf javierbits/Rumble-2-Flash-GGUF:Q5_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf javierbits/Rumble-2-Flash-GGUF:Q5_K_M # Run inference directly in the terminal: llama cli -hf javierbits/Rumble-2-Flash-GGUF:Q5_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf javierbits/Rumble-2-Flash-GGUF:Q5_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf javierbits/Rumble-2-Flash-GGUF:Q5_K_M
Use Docker
docker model run hf.co/javierbits/Rumble-2-Flash-GGUF:Q5_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use javierbits/Rumble-2-Flash-GGUF with Ollama:
ollama run hf.co/javierbits/Rumble-2-Flash-GGUF:Q5_K_M
- Unsloth Desktop
- Docker Model Runner
How to use javierbits/Rumble-2-Flash-GGUF with Docker Model Runner:
docker model run hf.co/javierbits/Rumble-2-Flash-GGUF:Q5_K_M
- Lemonade
How to use javierbits/Rumble-2-Flash-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull javierbits/Rumble-2-Flash-GGUF:Q5_K_M
Run and chat with the model
lemonade run user.Rumble-2-Flash-GGUF-Q5_K_M
List all available models
lemonade list
- Atomic Chat
Download bench_select.py from javierbits/Rumble-2-Flash-GGUF: direct link, hf CLI and curl.
- Browser
- Download file 2.67 kB
-
https://huggingface.co/javierbits/Rumble-2-Flash-GGUF/resolve/main/bench_select.py
- Command line
-
hf download hf://javierbits/Rumble-2-Flash-GGUF/bench_select.py
-
curl -L -o bench_select.py https://huggingface.co/javierbits/Rumble-2-Flash-GGUF/resolve/main/bench_select.py
2.67 kB
| #!/usr/bin/env python3 | |
| """Select a deterministic benchmark split from the FLEURS test sets. | |
| Requires data/data/<lang>/{audio/test.tar.gz,test.tsv} (see README download step). | |
| Writes data/bench_clips.json and extracts audio to data/bench_wav/<lang>/. | |
| .venv/bin/python bench_select.py --clips 100 | |
| """ | |
| import argparse, json, random, tarfile | |
| from pathlib import Path | |
| BASE = Path(__file__).resolve().parent / "data" | |
| DEFAULT_LANGS = ("en_us", "es_419", "pt_br", "de_de", "fr_fr", "cmn_hans_cn", "yue_hant_hk") | |
| def main() -> None: | |
| ap = argparse.ArgumentParser() | |
| ap.add_argument("--clips", type=int, default=100, help="utterances per language") | |
| ap.add_argument("--langs", default=",".join(DEFAULT_LANGS)) | |
| ap.add_argument("--seed", type=int, default=1234) | |
| ap.add_argument("--out", default=str(BASE / "bench_clips.json")) | |
| args = ap.parse_args() | |
| out = {} | |
| for lang in args.langs.split(","): | |
| tsv = BASE / f"data/{lang}/test.tsv" | |
| if not tsv.is_file(): | |
| raise SystemExit(f"missing {tsv} (download the FLEURS test split first)") | |
| rows = [] | |
| with open(tsv, encoding="utf-8") as f: | |
| for line in f: | |
| parts = line.rstrip("\n").split("\t") | |
| if len(parts) < 7: | |
| continue | |
| _, fn, txt, _, _, nsamp, _ = parts[:7] | |
| dur = int(nsamp) / 16000 | |
| if not (3.0 <= dur <= 25.0) or len(txt.strip()) < 10: | |
| continue | |
| rows.append({"id": fn[:-4], "file": fn, "dur": dur, "transcription": txt}) | |
| random.Random(args.seed).shuffle(rows) | |
| seen, sel = set(), [] | |
| for r in rows: | |
| key = r["transcription"].lower() | |
| if key in seen: | |
| continue | |
| seen.add(key) | |
| sel.append(r) | |
| if len(sel) == args.clips: | |
| break | |
| out[lang] = sel | |
| print(lang, "candidates", len(rows), "selected", len(sel)) | |
| for lang, rows in out.items(): | |
| dest = BASE / "bench_wav" / lang | |
| dest.mkdir(parents=True, exist_ok=True) | |
| wanted = {f"test/{r['file']}": dest / r["file"] for r in rows} | |
| with tarfile.open(BASE / f"data/{lang}/audio/test.tar.gz") as t: | |
| for m in t: | |
| if m.name in wanted: | |
| wanted[m.name].write_bytes(t.extractfile(m).read()) | |
| for r in rows: | |
| r["audio"] = f"bench_wav/{lang}/{r['file']}" | |
| r.pop("file", None) | |
| r.pop("dur", None) | |
| Path(args.out).write_text(json.dumps(out, indent=2, ensure_ascii=False)) | |
| print("wrote", args.out) | |
| if __name__ == "__main__": | |
| main() | |